Top 10 Reasons to Use EssayEdge Editors Samples of Our Work 100 Success Stories Acceptance Letters Our Business Principles Press & Industry Reviews 2005 Success Survey Results
	B-School Success (Admitted to McCombs) "I got my first admission from McCombs Business School at UT Austin this Wednesday. I appreciate your great help on my essays and detailed comments. The essays you edited for me were far more persuasive and sophisticated than I expected them to be." More...

What's in a Number? A Ranking Roundtable
Content provided by mbajungle.com

In the wake of Business Week's rankings, the answer is, "quite a bit." Job prospects, student morale, even administrators' careers are affected by the scores. But how useful are they? And what's their impact on the classroom? A roundtable discussion.

Are you number one? Number 21? How much difference is there, anyway? Whatever significance you may assign the rankings, this much is certain: There's no ignoring them. As schools vie for top applicants, ratings published by Business Week, U.S. News & World Report, The Financial Times, and others have become a critical compass for prospective students. A top ranking can be a financial boon to schools, which see applications and endowments swell as they ascend the ladder. But a slip in the standings can be devastating -- student morale dips and administrators can find themselves under fire, or just plain fired. Shortly before the release of this year's Business Week rankings, MBA Jungle assembled a panel in New York City to discuss the rankings' benefits, shortcomings, and continuing relevance in the New Economy. 

THE PARTICIPANTS

1. Michael Barclay: Finance professor at the University of Rochester's Simon School of Business. In 1994, Barclay was named to a Business Week list of the most popular B-school professors. Simon was ranked 21st this year.
2. Elizabeth Gabbay: Director of MBA recruiting for Scient, a two-and-a-half-year-old Internet consulting firm whose MBA recruiting targets have quadrupled since 1999.
3. Mary Giannini:  Director of campus recruiting of the tri-state region for Deloitte & Touche.
4. Tracy Lloyd-McRae:  A 2000 graduate of Georgetown University's McDonough School, ranked 26th this year.
5. Jennifer Merritt:  The editor in charge of the rankings issue for Business Week. The 2000 rankings are the first she's overseen.
6. Jessica Portner:  MBA Jungle moderator
7. Jonathan Shapiro:  A student entering his second year at the University of Pennsylvania's Wharton School of Business, ranked first by Business Week.
8. Fallaw Sowell:  Director of the MBA program at Carnegie Mellon University, ranked 14th.
9. Robert J. Swieringa:  Dean of Cornell University's Johnson School of Management. Cornell dropped to 18th in Business Week's 1996 rankings, then bounced to 8th, where it has remained since 1998.
10. Joe Tracy:  Vice president of the Federal Reserve Bank in New York and a former professor of economics at Yale and Columbia universities. In 1997, Tracy co-authored a study that was critical of rankings methodologies.
    

THE MEASURE OF ALL THINGS? 

Jessica Portner: I'd like to begin by having Jennifer Merritt tell us a little bit of the history and rationale behind the rankings.

Jennifer Merritt: The rankings were started 12 years ago by John Byrne, a senior writer at Business Week. He thought the rankings that were out there were inadequate because they largely relied on test scores, yields, GPAs, etc. He didn't think that was a good measure of whether a business school was doing its job. He felt the only way to truly measure a school was by talking to its customers -- recruiters, and students in their graduation year. 
Portner: Mr. Tracy, you co-authored a paper that was critical of rankings methodology. Tell us about your paper and its findings. 

Joe Tracy: The work was done jointly with Joel Walford while we were both at Yale. Joel is now at Wharton and I'm at the Federal Reserve Bank. It was a nontraditional research topic for two economists, and it came about mainly because our office was down the street from the Yale School of Management. We sensed the trepidation each time the administration waited for the rankings to come out. Since we were both educators, we cared very much about the learning -- what actually takes place once students are there. We decided to determine whether the rankings reflected the value added by the education, or whether they were more a measure of the students the program attracted. In other words, did they reward the schools that educated most effectively, or just the ones that admitted the brightest students?

The result was that both the Business Week and the U.S. News & World Report rankings were more closely related to the quality of the students coming in than to our measure of the value added.

Merritt: I'm curious about that, because the one thing Business Week really focuses on, which will often send schools up or down, is the quality of teaching, and whether students feel they're getting from their education what they need in the real world, which I would think is a value added. 

Jonathan Shapiro: But a lot depends on when you ask. I just finished Year One a couple of months ago. Before I started my summer internship, I would have said, "You know, a lot of the classes we're taking aren't responding to the changing economy."

Right now, I'm working in an Internet group, and I'm spending my time writing about supply-chain management and marketing-program mixes. So if you asked me now, I'd say, "Yes, the classes were great preparation." Next June, right after I graduate, I may have another answer in terms of the added value I got from the program. Maybe it would be better to ask students three years later, rather than right after graduation, as they're about to leave for, you know, a trip around the world. 

Merritt: This is something that comes up a lot actually -- when to measure? But with the graduates' having so much more work experience when they go in -- an average of five years -- we feel we can be confident in their responses about what they're learning. Also, the recruiters have experience with many schools over time. 

Tracy: Recruiters do have several schools where they interview, so they can make direct comparisons. The concern Joel and I had about asking students -- really at any point -- is that in most cases they've gone to one business school. Yet you're asking them, in effect, to make a comparison across a set of schools.

SUDDEN IMPACT

Portner: What do the rankings mean to each of you? What kind of impact do they have on students -- and prospective students? 

Tracy Lloyd-McRae: I definitely looked at the rankings. I know my classmates did, too. And we watch them. In an ideal world you want people first to think about what they want out of an MBA and look at the different programs and try to ask, "Is this the right city for me and for the career that I'm in?" and then use the rankings as a tool to find a school that's the right fit. Georgetown's ranked, I think, 25th or 26th by Business Week, so we see it as a challenge. The faculty and administration are doing a lot because they know we watch the rankings, and that recruiters do as well.

Mary Giannini: From a recruiter's perspective, I think we need to have a strong sense of the competencies and skill sets that we're looking for. You may not get what you need out of the top five schools, given the particular slant of the curriculum, where the students want to go after they graduate, etc. I've always been a real believer in the idea that first you should take a look at what kinds of core competencies and specific skills you need, and then you go and see where that fit is. 

For example, there's a small program at SUNY Albany, which is basically a regional MBA program. I don't even know if it's ranked at all. Yet if you're looking for someone with a concentration in human-resource information systems, which is a very niche-type program, they probably have the best one around. I've worked for two firms that do consulting for our clients on this very issue, so you bet we go up there. Now, do we look for people in their finance program? No. 

Elizabeth Gabbay: At Scient, we're in a very interesting position. We're only two and a half years old, and we're just starting to populate the company with MBAs. This is going to be a pivotal year for recruiting because we've just exploded -- our MBA recruiting targets have quadrupled from last year. But we don't really have a benchmark yet as to what will be successful, which is why I think the rankings will be a very interesting tool for us. When I was at Andersen Consulting, there was a commitment there to using the rankings and hiring from the business schools that they were comfortable with. That made it a little easier to do recruiting, of course, but you were sort of locked into and very driven by the rankings. 

Portner: Dean Swieringa, when you came to Cornell in 1997, to what extent did you use the rankings to identify those areas you might improve upon?

Robert J. Swieringa: I won't downplay the role that the rankings had in 1996, when the school dropped dramatically. In many ways that was a wake-up call, and the school, the students, and the faculty realized that they were in a very competitive world. They really had to sit back and think about what they wanted to be in the future and what kinds of investments they were willing to make.

Michael Barclay: As a professor, it's been my experience that the rankings have had a very dramatic effect on what happens in the classroom. Some of the changes have been very good for students; others not as good. There's intense competition among business schools to make students happy these days: We're going to put a lot of effort into our teaching. We're going to make sure that our lectures are well prepared and organized. We're going to use computer-generated graphics. We're going to try to put you in a building that's comfortable, has state-of-the-art facilities, good acoustics, and sight lines and everything else.

But if there's a downside to the competition to make students happy, it's this: Students are very good at judging the style of the presentation in the classroom; but at that stage in their career they're not in the best position to judge the content of what's delivered. And I think there's been a disturbing trend in business schools over the last 10 years or so in that part of the product has gotten watered down.

Rather than pushing students to do difficult analytical material, and making them struggle with that, sometimes the instructor will say, "Well, let me give them a little bit of intuition and we can talk about the problem and think about what the solution would look like. They'll be happier. I'll be happier. My teacher rankings will go up. Everything will look good." 

I think at the Simon School the students have benefited from the rankings because we've put a lot of effort into our teaching, and we have the checks and balances in place to make sure our program is as rigorous and challenging as possible. And we're going to do that regardless of how it affects the survey. I'm not convinced every other top business school can make that claim.

Swieringa: I don't necessarily accept the premise that academic standards are dropping. Some of our most demanding courses are taught by our most effective teachers, in terms of the kinds of evaluations they're given. Any change in style at Cornell is really being brought about by the marketplace. Students are different today. They have an experience base that's quite different than what we had years ago. We're trying to draw out that experience and take advantage of it in the classroom. There's more pressure on the professor to try to indicate the relevance of what's being taught.

Barclay: I certainly agree that the best teachers at any university are also going to be delivering the high-quality, rigorous content, and, you know, I don't want to push this panic button about the demise of MBA education. I think the quality of education is still very high in almost every business school, but it has changed. 

For example, there's this currently fashionable notion of "hitting the ground running" when you get to your job. That concept can be overdone, because there's a fundamental difference between training you for your next job and providing you with an education that's going to be valuable throughout your career. Of course, it would be very easy for us to train the students to do their next job. But we really have to think about all the skills a person needs in order to have a successful career. What we'd like to do is specialize in the broad, general education that's going to set the student up for a very successful long-term career path. 

Swieringa: But I don't think it's a choice between less filling and tastes great. Let me give you an example. We have the Parker Center, an electronic platform with a group of professors -- Charles Lee, Swaminathan, and others -- who teach a very analytical approach to the analysis of financial statements and financial information. The electronic platform allows us to teach courses in analysis, building portfolios, doing modeling exercises, and so forth using point-and-click technology. We have data feeds -- we can follow 14,000 U.S. companies and 85,000 international companies, almost all with point-and-click technology. So our students can learn how to implement and to make the choices that are guided by the analytics. It's not one versus the other. 

Fallaw Sowell: It's a subtle point, but I think where you may be seeing the softening of the business school curriculum is in the required courses. Absolutely the electives are rigorous and demanding, but the students are the ones deciding whether or not they actually take those classes. So what you're seeing is a softer side on the required courses that everybody has to take. That's what goes on. Again, this may be just the result of heterogeneity in the student body. It may not be a dumbing down as much as adjusting for a different audience. 

PLAYING HARDBALL 

Lloyd-McRae: I think it's important that professors be responsive without dropping the quality. And that's really not easy to do when you've got students who are bleary-eyed and complaining. At Georgetown, we went from a traditional schedule to six-week modules, so that we now have 10 classes a semester. This was after the faculty and administration analyzed other MBA programs. But when they made the change the first time, with our class, it was grueling. The professors wanted to know what we thought: "Okay, this is the first time we're doing this so we need your feedback." And we're like, "We're not getting sleep." 

Some of the professors said, "You're not supposed to sleep. This is MBA school." Others listened and adjusted, and it was still rigorous. The good professors, the ones we rank well, listen, laugh at us, and adjust where they can; but at the end of the class, when we've gotten some sleep, we wake up and think, "You know, I really learned something."

Shapiro: There are professors, I'm sure, who go too far and say, "Okay, instead of running all the numbers on this case, I'm going to make your life easier and force-feed you some of the answers."

Barclay: That's a good example, because the way a case is going to work most beneficially is if the students take that mass of details -- some relevant, some irrelevant -- and just kind of barrel through that and try to figure out what are the most important points. If you want students to do that, it takes training. The easiest thing for the instructor to do is say, "Okay, now you've all read the case and you've all thought about it, now how about this for an answer?" We'd go through and blah, blah, blah, and everybody says, "Oh yes, that would be great."

On the other hand, what you've just trained the student to do is read the case, think about what was in there, and then come out prepared to listen to the answer. And that's a much different process than training the student to say, "Let's struggle with this case for however many hours it takes so that when I'm done I'm confident that I have the right answer."

Lloyd-McRae: I think that the kind of students who are attracted to the top business schools are not looking for that. Sure, you may hear them complain about the workload; they may be ticked off that they can't eat a meal at home; but if you did it the easy way, the school would be penalized. The students would say, "This is not what I'm paying for." In the end, they're very discerning customers.

Shapiro: Darden, at the University of Virginia, jumped up a number of notches a few years ago largely because of the rigorousness of the courseload. At the very end, though, you add those last couple of pieces that push people over the edge. You know, "Before, I could stay up really late and research companies that I wanted to interview with, and now I can't even do that anymore. And now I'm at a real disadvantage." And so the school shot up in the rankings but then slumped back down.

Lloyd-McRae: You go to Darden because you want that rigorous education. You know you're not going to sleep, but they saw that students appreciated that. The problem is, the administration responded by making it even tougher. They piled on too much, so the students didn't even have time to look for jobs. When recruiters came, they felt discombobulated because nobody was prepared for interviews. And that doesn't reflect the quality of the students.

Giannini: No -- I think there's a balance here, and obviously they're not balanced anymore. Why do people go to business school? I don't think they go because it's some kind of intellectual exercise. They go because it's going to improve their career opportunities. If you damage that piece of why they're going, you're not meeting the needs of the students. I've recruited at Darden, and, I mean, they walk in and they're zombies. 

Shapiro: There's no one from Darden here to defend it, so I will, I guess. I have friends from Darden who'll say, "It's the best preparation, and yes, I crashed while I was there, but it was well worth it." I also know people who've said, "I didn't need to go through all of that." 

FALLING FROM GRACE 

Sowell: All of the schools have responded to the rankings. A great change has been taking place in business school education. The quality of students has gone up dramatically. The quality of education has gone up dramatically. But when you go in and do a ranking, a top 20, and you force this curve, what you miss is the wonderful improvement that's taking place in this industry. It's the equivalent of everybody scoring above 90 on the exam and someone saying, "Well, sorry you got that 91. You get a C." In the end, they're giving a failing grade to some people by forcing a rankings curve. 

Merritt: You mean improvement based on starting salaries or -- I'm unclear where this comes from. 
Sowell: We can certainly look at salary. We can look at GMAT scores and other things. What a ranking misses is the overall quality improvement that's been taking place over time. People think, well, if you drop four rankings, then you must be doing something wrong. Everybody is just doing great stuff and that gets missed when we start getting down to second-guessing -- you know, "They dropped two spots."

Tracy: Look at GMAT scores [which U.S. News uses in its rankings, but Business Week does not]. There's a huge improvement across the board. It's like the Olympics, where the difference between first place and 13th place in the 100-meter race is three-hundredths of a second. 

Barclay: But does that mean that we've got a huge number of smarter people now? There are some schools that will say, "Go do the review course," and so people will take the GMAT multiple times. That's a good argument, in my mind, against rewarding schools in the rankings for having high GMAT scores. 

Merritt: And I'd be interested to see how important that is for the New Economy worker, anyway. I mean, I wonder what Bill Gates would have scored on his GMAT? In any event, we always point out that there's not a lot of difference between top schools. But I can reasonably say that there are some huge gaps overall, based on what students and recruiters are saying.

Tracy: I think, though, that even if you qualify the rankings, there's a false sense of precision. If we take the same school, take the same quality students, and run the whole year over again; bring in recruiters from the same campuses and then reinterview them; and do that 1,000 times, you're going to find that these rankings jump all over the place. 

Merritt: It's not perfect, but it's the best we feel we can come up with. 

Tracy: Well, it's devastating if you fall out of the top 20. 

Merritt: Yes, there are schools out there that are using this in the wrong way. And I have no respect for them. They look at the rankings and say, "Oh, my God, we have to get up in the rankings. What do we have to do? Well, you're fired and we're doing this." That response is such a disservice to the students. And you know what? Those schools often go down. You know who they are -- everyone knows. I'll be the first to admit that this is not perfect. But we feel that if you look at the choices we have, this is the best way we've come up with. 

Portner: At Business Week, do you ever feel responsible or do you ever cringe when you hear about people that are let go because the school went down in rankings? 

Merritt: I don't feel responsible for their being fired. I feel sorry for the school taking that tack. Every dean from probably the top 30 or 40 schools, sometimes even more, comes in to talk to me, and this is one thing I tell them: I feel extremely responsible for what happens, and therefore I am committed to doing this the best that I can. And so I hope that you use the rankings for good and not for evil. Maybe you totally ignore them, like Darden. They didn't totally ignore them, but to some of that stuff, they said, "You know what? That's too bad. This is what we are and this is what we offer." 

INTO THE FUTURE 

Portner: Let's talk a bit about the future of the rankings. Are there ways you think the rankings could -- or should -- evolve? 

Lloyd-McRae: I think the rankings could offer more information so they're not taken so literally. I think they're a necessary evil. They raise the standard by making us compete. We're getting a new building at Georgetown, and it's because the administration looked at other schools. The rankings encourage that. 

But it would be nice if the rankings did have a much more detailed breakdown as to which schools do what best, and more about the culture. For example, in the survey I got, I had to write some things I thought were really important in the little comment box at the bottom. We had international residency required for all of our students. Now where do I put that in a survey? That was one of the highlights of my time at Georgetown. 

Merritt: I don't know if you've seen our rankings book or the Business Week Web site, but we put those there. We do try to break it down as much as possible, but in a magazine you have limited space. 

Gabbay: Rankings are a valuable tool, but I think there's still something nebulous we'll never get from any ranking. We had a student, for example, who was very bright, from a top five school; and as time went on, we found that culturally this was somebody who was going to clash, so we retracted the offer. There are some things you can't capture in a survey. 

Sowell: I think it will meet the needs of this industry better when we get away from rankings and into profiles. What seems to strike a chord is the idea of people using rankings to find the appropriate fit for them. Every school has its own personality, and when someone's looking to apply, they need to find out what's right for them given their background and aspirations. And a ranking doesn't necessarily do that, because it's just one measure that someone has written down. Some algorithm. 

A student says, "Hey, I want to go to business school." That carries opportunity costs. She's walking away from her salary, spending on tuition, moving someplace, living. This is a huge expense. I know that numbers are appealing, but others are stepping into the marketplace, like The Financial Times Web site, where you can actually design your own rankings. This is how the industry is evolving, and, I think, should continue to evolve. 

Swieringa: I don't know what the future is going to be like, but the business world is changing dramatically. We in academia have traditionally focused on accounting and finance and marketing, but increasingly the emphasis is on leadership, technology, globalization, and entrepreneurship. People coming to school now may also have experience in the New Economy. So I wonder if the role of the organizations that offer rankings will be in providing information and tools that will allow people to do the dynamic analysis you were describing. 

Merritt: One difference that I can see in future rankings is the addition of some kind of measure of the influence of a business school on the real world. We're struggling to figure out how that would work, but we think that's important. It would measure rviettle bit of innovation, a little bit of how a school is actually affecting the world around it -- not just the world it's in. 

Barclay: That's a little bit ironic, because we were talking earlier about how the rankings affect faculty members and what goes on in the classroom. And one of the traditional ways in which business schools have influenced the outside world is with their fundamental research. If you think about the finance area that I'm in, you don't have to go very far back to find things that revolutionized business practice -- the Capital Asset Pricing Model, the Black-Scholes options-pricing model -- and one of the things the rankings have done is to shift the set of resources in business schools away from fundamental research toward teaching, infrastructure, and technology. In the short run, all of those have been very good for students. I get concerned, though, that the shift is consuming our capital when we still need to understand the world better and change business practice. 

Tracy: I think we've all agreed there's no ideal ranking. The real key is giving prospective students information and helping them process it. What was alarming to Joel and me when we started surveying schools was that there really is no standard for certain basic data items that every school agrees on. We'd get call-backs from schools saying, "Well, how do you want this number reported? We give it this way to this group, and other groups want it this way." That is very confusing to students, because then you're comparing apples and oranges. 

Barclay: In the end, this whole thing boils down to supply and demand, right? If we could get all the students we wanted, then none of this would be an issue. We'd say, "I don't care what our ranking is." 

Back To Top