Microsoft Corp. did not name a successor, but the company said it is forming a search committee, which will include Gates. Ballmer will stay on until a replacement is found.
"There is never a perfect time for this type of transition, but now is the right time," Ballmer, 57, said in a statement released by the Redmond, Wash., company.
After the news broke, Microsoft's stock shot up as much as 9 percent shortly after the markets opened. The shares came within two dollars of their 52-week high.
Microsoft has struggled in the Ballmer era. When he took the helm in January 2000, Microsoft's market value stood at more than $601 billion. Today, the company is worth less than half that amount, at nearly $270 billion.
The CEO's announcement comes less than two months after the company unveiled a sweeping reorganization of its business in an attempt to reignite competition with faster-moving rivals such as Apple and Google.
Among Ballmer's biggest mistakes, detractors say, were his initial dismissals of emerging threats from Google and Apple. He consistently pooh-poohed Google as a one-trick company during its early years and in 2007 declared: "No chance that the iPhone is going to get any significant market share."
Google quickly made important inroads in Internet video, online maps, email and mobile computing and contributed to the damage that the iPhone and iPad have done to Microsoft and its partners in the PC market.
Microsoft, along with other companies that thrived in the era of personal computers, is scrambling to transform its business as people increasingly come to rely on smartphones and tablets.
Although it derives some three-quarters of its revenue from sales of software and services to businesses large and small, Microsoft has failed to capture the imagination of consumers who have become more enamored with mobile gadgets than PCs. Response to the newest version of its flagship Windows operating system, Windows 8, has been lukewarm.
In his statement, Ballmer noted that the company is moving in a new direction and needs a CEO that will be there for the longer term.
Microsoft, he added, "has all its best days ahead."
Ballmer met Gates in 1973 while they were living down a dormitory hall from each other at Harvard University. He joined Microsoft in 1980 to bring some business discipline and salesmanship to a company that had just landed a contract to supply an operating system for a personal computer that IBM would release in 1981.
Ballmer, a zealous executive prone to arm-waving and hollering, did the job so well that he would become Gates' sounding board and succeed him as CEO in 2000. He has worked at Microsoft for 33 years, matching the tenure of Gates, who left the company in 2008.
"It's a tad surprising, but every other business head has been rotated out," said BGC Financial analyst Colin Gillis. "They swapped out all their segment heads over the past few years. The only one they haven't changed is the CEO."
Though investors cheered the news on Friday, Gillis cautioned that it could be a "tough 12 months" for the company.
The obvious successor - former Windows head Steven Sinofsky - got booted by Ballmer, he said.
Sinofsky left the company shortly after the launch of Windows 8 last year. He recently announced that he joined the venture capital firm Andreessen Horowitz.
Veteran executive Julie Larson-Green, the head of Microsoft's devices and studios engineering group, has been floated as a potential successor. She was promoted to her most recent position in July, after being tapped in November to lead all Windows software and hardware engineering.
Although the company said Friday that it will consider both internal and external candidates, some analysts are betting that the company's next leader will come from outside.
Walter Pritchard, an analyst with Citi Investment Research, said Microsoft's expected focus on external candidates will make it tough to predict who will become the next CEO and what direction they will take it in. He added that the search will likely take a significant amount of time, potentially the entire 12 months Ballmer has said he will stay.
When Ballmer joined Microsoft in 1980, was populated with geeky programmers, led by Gates and the other founder, Paul Allen. Ballmer had already held a product management job at Procter & Gamble and was attending Stanford University's graduate school of business when Gates convinced him to move to the Seattle area to whip Microsoft into shape.
Ballmer dropped out of Stanford, but only after Gates agreed to give him an 8.75 percent stake in the then-tiny startup that still hadn't even incorporated as a company. It turned out to be one of the world's greatest business partnerships. By late 2012, Ballmer had accumulated an estimated fortune of nearly $16 billion from his initial Microsoft stake and additional stock options he later received.
He also was instrumental in growing Microsoft from a company that had fewer than 40 employees and $12 million in annual revenue when he came aboard. In 2012, Microsoft had 94,000 employees and $74 billion in annual revenue.
When he took to the stage to extol Microsoft, Ballmer often acted more like a crazed cheerleader than the chief executive of an influential company. In one presentation that eventually became a viral sensation on the Internet, Ballmer bounded across the stage, jumping up and down while yelping and imploring the audience to stand up, before breathlessly proclaiming, "I LOVE THIS COMPANY!"
But Microsoft enjoyed its greatest success with Gates at the helm and Ballmer as his sidekick.
Gates turned over the reins to Ballmer in January 2000 in what was considered to be a surprise move, because Ballmer had been considered more of a numbers and sales specialist, not a technology specialist.
The CEO change came just a few weeks after Microsoft's stock hit a record high of nearly $60, on a split-adjusted basis.
Janney Capital Markets analyst Yun Kim said that while the stock should get a boost from the news, investors shouldn't get too excited, because the company itself won't change overnight.
Kim said the new CEO, who will likely come from outside the company, faces the "daunting task" of making Windows relevant amid the continued consumer shift away from PCs.
Part of the Microsoft's downfall stemmed from the bursting of a technology bubble that helped inflate the company's stock just before Ballmer took over.
But Microsoft also has fallen out of favor because many investors had concluded Microsoft was interested in protecting its Windows franchise than coming up with new ideas and products to enter promising new markets on the Web and mobile devices. It didn't help that Ballmer seemed to initially underestimate the threats posed by a resurgent Apple Inc. and Internet search leader Google Inc.
By the time Ballmer took Google more seriously and began pouring money into trying to build a better Internet search engine, Microsoft already was hopelessly behind. The company's online division lost billions of dollars without putting a serious dent into Google's dominance of the field.
Google's rise riled the quick-tempered Ballmer, especially when key Microsoft engineers began defecting to the then-smaller company. After one Microsoft employee met with Ballmer in November 2004 to tell him he was leaving to join Google, Ballmer threw a chair across his office, according to a sworn declaration filed in a lawsuit. Ballmer then launched into an obscenity-laced tirade In which vowed to "kill" Google.
By 2012, the iPhone was generating more revenue than Microsoft was as an entire company and giving people less reason to replace their PCs. Again, Ballmer had to scramble in an attempt to adapt and ordered a dramatic makeover of Windows so it could run on mobile devices, as well. The new system, Windows 8, borrowed many of its ideas from the software that ran the iPhone, just as Microsoft had copied some of the concepts for its early versions of Windows from Apple's Macintosh.
The iPhone's immense popularity helped Apple overtake Microsoft as the world's most valuable company while Ballmer was CEO.
In midday trading, Microsoft shares rose $1.88, or 6 percent, to $34.27. Over the past 52 weeks, the company's shares have traded between $26.26 and $36.43.
AP Technology Writer Michael Liedtke in San Francisco contributed to this story.