Softpanorama Russian Software Bulletin, 2002

All Eyes on Google

Google was launched less than four years ago by two graduate students in computer science: one, a Russian émigré named Sergey Brin, now 29; the other, a Michigan-reared engineer named Larry E. Page, now all of 30. As a gateway to 3 billion Web pages, Google is a strangely unadorned site: 37 words, four tabs and a blank space where you type in a query of up to 10 words. Google's over 10,000 networked Google computers crawl through an index to those 3 billion pages, rank them with an equation that includes 500 million variables and spit out up to a few thousand listings. The ranking takes 500 milliseconds; the computers can handle a peak rate equal to 7 million queries per hour.

But Google has become much more than merely a search service. It is a daily tool and main entry point for millions of users, stealing the spotlight from the browser (Explorer or whatever) and Internet portals like Yahoo. It is a labor of love for programmers, who have built applications off of Google and posted them like trophies on the Web. One does a "smackdown," comparing the Internet ubiquity of two words ("love" beats "money," but not by much); another creates poems (see boxes).

For Wall Street and Silicon Valley, Google is the great bright hope for an initial public offering that might revive moribund tech stocks. And Google has become its own meme, the stuff of New Yorker cartoons and a brand, like Kleenex and Band-Aid, that is in danger of becoming a part of the English language. You don't search for something on the Web anymore. You Google it.

Google now can be queried in 36 languages, with more to come. At the posh Hotel Bel Air, in Los Angeles, manager Lisa Hagen makes a point of Googling all guests before arrival, searching out better ways to spoil them. "If we find out they like to jog early in the day, we make sure they get a room with morning sun," she says. In Boston, Mark Kini manages a small limousine service that spends 80% of its ad budget on Google and other search sites. Says he: "It's how we survive the recession." In Westport, Conn. consultant Elena Amboyan's kids use Google daily; even when they research something at the library, they say they're Googling it.

It is all much more than Brin and Page ever had in mind when they started. "Sure, I'm surprised by the success," says Brin, unassuming, rumpled and wiry, his sneakers scuffing the upholstery of a conference-room chair. Users love Google, he says, because they find things there when they are desperate to know an answer. Keep offering better results and you hold their loyalty forever--and sell them stuff. Page adds that Google has become "like a person to them, helping them and giving them intelligence any hour of the day."

The passion and success igniting Google, and its emergence as a new interface for the Internet, have made it a rich, fat target for rivals. Yahoo (NasdaqNM:YHOO - News) is taking aim. So is the biggest search outfit, Overture (NasdaqNM:OVER - News), a little-known billion-dollar vendor that provides unbranded search services for other Web sites and has sued Google, alleging patent infringement. A gaggle of some 200 Web sites in China is reportedly going after Google, too.

And now Google faces the most lethal threat of all: Microsoft (NasdaqNM:MSFT - News), aroused, is taking aim at the popular site. This bears an eerie resemblance to the rise--and calamitous fall--of Netscape, the first commercially successful Web browser.

Will Google be the next victim of a Windows that swallows everything? To help ensure a future, Brin and Page brought in a grown-up as chief executive, Valley veteran Eric Schmidt, 48. Fittingly, Schmidt had abundant experience struggling against Microsoft in his two previous jobs: He was chief technology officer at Sun Microsystems (NasdaqNM:SUNW - News), then chief executive of Novell (NasdaqNM:NOVL - News), two companies that thought, wrongly, they had Microsoft licked. Google's founders credit Schmidt with successfully managing their company's most intense period of growth.

To survive and succeed will require lots of talent, lots of acquisitions and lots more money. More important, Google will need to quell the hubris that is much in abundance at the jubilant company these days. To be at Google is to bask in your own public relations. The hallways of the company's four buildings in Mountain View, Calif. are decorated with articles from around the world praising the company. One current job posting includes duties as Google's company historian. Over 70 of the 800 employees have Ph.D.s. Google's head of engineering admits his big-brained staff is in awe of itself; he hopes the simplicity of the Google page masks that from the outside world.

In some ways Google feels like the giddy dot-coms of the stock-market bubble, circa 1999. Informal to a fault, Google offices are littered with party-colored lava lamps, bins of free Coke and candy and giant plastic balls that invoke Google's multicolored logo. The cafeteria serves free lunch to the workaholic ranks (and dinners, too; there's lots of code to write). When pizza gets delivered at one o'clock in the morning, plenty of people are on hand to devour it. Every day a thousand more résumés arrive from people hoping to join this work party.

But the dot-com parallels end when you look at the finances. The dot-bombs burned through tons of other people's money. Google makes a pile of cash on its own. After it went live in September 1999--six months before the Internet bubble finally popped--Google took in perhaps $25 million in 2000. Then it leaped fourfold to approach $100 million in 2001 and tripled to $300 million last year. Its gross could more than double this year to $700 million, estimates Safa Rashtchy of U.S. Bancorp Piper Jaffray.

Google, privately held--and determinedly so, for now--won't talk numbers, but it does brag that it just logged its ninth consecutive profitable quarter.Its revenue flows include ads (the bulk); search services for Yahoo, America Online and other sites (perhaps $100 million there); and custom-tailored, bright yellow servers for corporate accounts.

"Cheesy as it may sound," says cofounder Brin about the company's early days, "we never thought in terms of revenue streams." Now he must, for the next year or two could determine whether Google delivers on the high hopes it inspires in so many quarters or instead falters, glorying in its early success while others plot its doom.

Google traces back to 1995, when Sergey Brin and Larry Page, whose fathers taught college math, met at Stanford. The sons saw search as an interesting problem in organizing very large datasets.

At the time, users typed in a few words and got a list of thousands of Web sites using those words, but most of the results were irrelevant. Brin and Page quelled users' frustrations by adding order to this randomness. They judged a listed site's prominence by how many other Web sites valued it enough to have links to it. They gave sites a resulting "Page rank" (for Larry, not Web pages). This cliquey if democratic approach was later augmented by other algorithms that weight sites by other variables--news sites get a higher ranking than a 16-year-old's personal Web log.

The two grad students soon found their results were a step above any other kind of search. They had dubbed this system Back Rub, after the "back links" that pointed to a site. They adopted the name Google in early 1997, in a tribute to scale, a play on the number known as a googol--a one followed by a hundred zeros. The universe does not contain a googol atoms. The denizens of the company headquarters breezily refer to it as the Googleplex, that being the word for the unimaginably large number defined as a one followed by a googol of zeros.

Brin and Page introduced Google to the world in a paper they presented at the World Wide Web Conference in April 1998. Naively, they were downright hostile to advertising, calling it "insidious … because it is not clear who ‘deserves' to be there, and who is willing to pay money to be listed." A few hundred million in revenues later, Brin has changed his mind. On a Google results page, he says, "There are eight spots for ads and ten search results. It's a lot of room for diversity."

Soon after, the pair began trying to sell their technology to Web sites, including Infoseek, Excite and Yahoo. They found no takers; one chief executive told them that if his site could search only 80% as well as everyone else's, that was okay by him. "That company is now out of business," Page says. Then their faculty adviser invited them to a breakfast with Sun Microsystems cofounder Andreas Bechtolsheim on the Stanford campus. Midway through the demo, Bechtolsheim stopped them and wrote a check for $100,000 to Google Inc.

This presented a problem, as Google didn't yet have a bank account. There wasn't even a "Google Inc."--they hadn't yet decided to form a company. The check sat in a drawer for several weeks, and then they got serious.

By June 1999 Google had raised almost $30 million from venture firms Sequoia Capital and Kleiner Perkins Caufield & Byers, plus Stanford and individual investors. Three months later the Google site officially blasted off. It could scan 30 million Web pages. Today it culls 100 times as many, and still taps only half the Internet; the rest lies behind corporate firewalls or in isolated islands unlinked to anything else.

As Google began to thrive, the Web world was crashing, and this, too, proved lucky for the pair of founders. As dot-coms collapsed, Google took over cheap office space, barely used Aeron chairs, dozens of servers and platoons of out-of-work programmers. By mid-2001 Google was profitable, employed several hundred people and was seeing traffic grow 20% every month. Thriving despite the surrounding downturn, Google went shopping for a seasoned chief executive. "My job was to impose a little order," Schmidt says now. "I made it clear that I wasn't coming in to get rid of the founders." Sergey Brin gave up his chairman mantle and assumed the title of president of technology; Page, who had been chief executive, is product president.

While the two techies concentrated on improving their search formulas, Schmidt focused more on building a better business model. Google had run ads with its search results for a while, but on a fixed-fee basis. Its main rival, Overture, publicly held and with $668 million in sales last year (it projects $1 billion in revenues this year), had already gone a step further. It exacted higher fees from advertisers by selling them rights to given keywords so their ads pop up first when those words are entered in a query. Sponsors paid on a cost-per-click basis instead of the usual cost-per-thousand-visitors.

At one point in 2001, Google officials even met with Overture to compare notes, Overture officials say. In December 2001 Google started a similar test on its Usenet section, unveiling a service called Adwords. The response was so enthusiastic that, by February 2002, Adwords had been extended to all Google listings. It grew to 100,000 bidders in ten months, and thousands more advertisers are still signing up. Total Web advertising fell about 5% last year, to $6.5 billion, while search ads almost tripled to $1.4 billion and could hit $7 billion in five years, says Piper Jaffray's Rashtchy. (Google itself advertises very little, instead relying on word of mouth.)

"Some companies have purchased thousands of keywords, and they use them to test multiple products against multiple words," says Sheryl Sandberg, director of the wildly successful Adwords program. Noting most all Adwords bidders are U.S.-based, while half of Google searches are by users overseas, Sandberg sees huge growth in foreign markets. "The monetization should follow. This is a global bid," she says. Ads are sold in 11 languages.

Schmidt calls the success of Adwords "a total accident--when we went off fixed pricing, my only directive was В‘Just don't let revenues drop.'" His foes at Overture allege instead patent infringement, suing Google in April of last year; one month later America Online (NYSE:AOL - News) dropped Overture in favor of Google Adwords. The case is likely to drag on for a long time.

In Adwords, businesses use an auction system on the Google site to bid for the most popularly searched words and phrases. Google gets paid every time someone clicks on the ad itself. Bids start at 5 cents per click but can go to $15 or more for high-end products like helicopter parts. Critically, Google demotes a sponsor to a lower rung on its page if its response rate is too low, elevating a rival's ad for getting more clicks. This imposes a built-in pressure on businesses. They're even asked to revamp wording if less than 0.5% of viewers click on their ads. By contrast, many traditional banner ads get click rates of just 0.3%.

This could transform the $193 billion business of direct marketing. Junk mailers constantly work on narrowing the recipient list to the people most likely to respond and on jazzing up the envelopes to trick them into looking inside. Google ends the guesswork. People directly declare what interests them, and Google feeds them an appropriate ad. The ad's few pitch words are critical. For big corporate accounts like Dow Chemical, Google account executives continually recraft the message, like a haiku of commerce, aiming to maximize the click-through.

Google's long-term dream is to index all of the world's public information and make it searchable--everything from driver records to radio shows and films--and reap profits from it. This is scarier than it sounds. Google holds an archive of 800 million postings to Internet newsgroups, from alt.sex.bondage to alt.humanities.classics, most of which it bought on the cheap just before Dejanews.com went out of business in 2000.

It is a strange bazaar of information and a repository of embarrassment for people who were forthright (or shortsighted) enough to forgo anonymity in their postings. Google easily unearths the Web's first mention of Microsoft; and Sergey Brin's 1992 complaint about selling his car; and the musings of a married midwestern academic who posted a plea on alt.sex.fetish.tickling. Ours for the ages, unless he follows Google's somewhat obscure directions--located in the "Groups Help" section--on removing work from the archive. Even posts like that one trigger a precision-targeted ad: One offers "Discount 14-K Gold Anklets." Like much of the Web, Google also makes good money on porn.

While Google wants to own the world, Microsoft is going after Google. It now has 70 engineers working on search technology, and by some accounts it could triple that staff. Its new best friend is Overture, which already provides search services for Microsoft's MSN online service. Overture scientists frequently visit Microsoft in Redmond to plan next-generation features. Microsoft also could acquire a search company this year; one likely candidate would be San Francisco-based Looksmart (NasdaqNM:LOOK - News). Neither Overture, with a market capitalization of $669 million, nor Looksmart, at $328 million, would be more than a bagatelle for Microsoft, which has $38 billion in cash.

The Google guys profess to be unfazed. They have assiduously avoided the sins of Netscape, which belligerently jeered at Microsoft's efforts to build a Web browser. "Netscape mooned the giant," says one Google exec, noting Google welcomes Microsoft ads on its site. Plenty of other threats abound. Yahoo, despite investing in Google and paying for its service, in December paid $235 million in cash to acquire faded search firm Inktomi. Overture recently spent $177 million for the Web-search assets of Fast Search & Transfer and AltaVista, while Ask Jeeves (2002 revenues $74 million, net loss of $15 million) put up $3.8 million for Teoma. Even Google's engineers admit Fast and Teoma deliver results comparable to theirs.

Google has bought some prizes of its own, including personalization technology that "learns" what you are interested in based on previous searches; and a company called Blogger, which helps people set up their own Web-based diaries, or Web logs. More "blogs" mean more content, yielding more pages on which to run ads and more links to other pages. The more links, the better Google's results. Most recently Google scored a company called Applied Semantics, whose content-scanning techniques can be used to tailor ads not just based on the words a user searches, but also on the actual pages he reads on the Internet. That buy was a double score for Google--Applied Semantics had been selling those services to Overture. In the week following the purchase, Overture's shares fell about 30%.

The need to acquire more tech could add to the pressure for Google to go public, so it could use its stock as currency. Both Brin and Page are daunted by the prospect of baring Google's secret financials and losing focus in the drive to boost profits every quarter. "I fear we'll grow shortsighted and lose the wider potential applications of our company," says Brin. "The biggest thing we'd lose is the opportunity cost of what we could do if we didn't go public."

But Google's growing ranks want it, Wall Street bankers yearn for it and clues hint that all of them will get it. In the overcrowded office of Sheryl Sandberg, the 33-year-old Adwords chief, sits a crimson lava lamp given to her by investment bankers at Morgan Stanley. Very hip, very Google-geist. The former U.S. Treasury official says with a laugh, "They have high hopes for us."

Downstairs, past a Google grand piano and a few big plastic balls, Chief Executive Schmidt convenes a meeting of two dozen managers for a project they refer to as "Keeping Eric Out of Jail." They are altering Google's billing and accounting systems to comply with the new Sarbanes-Oxley Act--a law that applies to all public companies but no private ones. It may take until October to comply, but Schmidt's urgency is palpable.

Every Friday he holds a companywide meeting, preaching to a cocky flock. Along with Brin and Page he talks business, technology--and attitude. He reminds these whiz kids to count on nothing. Remember the Netscapes, he exhorts, the high-tech stars that gained fans, made paper millionaires of the early staff and then burned up in the heat of competition. Just about everybody, save Google's massing rivals, hopes they're listening.

Three short notes on Russian Crypto History

• russ1
• russ2
• russ3.kript
• russ4.kript

Tetris is Hard, Even to Approximate

Authors: Erik D. Demaine, Susan Hohenberger, David Liben-Nowell
Comments: 56 pages, 11 figures
Report-no: MIT-LCS-TR-865
Subj-class: Computational Complexity; Computational Geometry; Discrete Mathematics
ACM-class: F.1.3; F.2.2; G.2.1; K.8.0

In the popular computer game of Tetris, the player is given a sequence of tetromino pieces and must pack them into a rectangular gameboard initially occupied by a given configuration of filled squares; any completely filled row of the gameboard is cleared and all pieces above it drop by one row. We prove that in the offline version of Tetris, it is NP-complete to maximize the number of cleared rows, maximize the number of tetrises (quadruples of rows simultaneously filled and cleared), minimize the maximum height of an occupied square, or maximize the number of pieces placed before the game ends. We furthermore show the extreme inapproximability of the first and last of these objectives to within a factor of p^(1-epsilon), when given a sequence of p pieces, and the inapproximability of the third objective to within a factor of (2 - epsilon), for any epsilon>0. Our results hold under several variations on the rules of Tetris, including different models of rotation, limitations on player agility, and restricted piece sets.

Full-text: PostScript, PDF, or Other formats

Notes on Virus Paranoya (Russian translation by Constantin E. Climentieff aka DrMad). Original English version is here

SourceForge russian - Foundry -- several projects including jedit

Tetris News  Pajitnov's version of Tetris creation. Pajitnov moved to the USA in 1991. Compare with Vadim Gerasimov's account  which I consider is more accurate... The Pajitnov's company was sometimes pretty aggressive in enforcing Tetris trademark and that damaged Pajitnov's reputation among developers. See song.li ... it's about the ego

By ALEX PHAM, [email protected]

One of the first games played on the personal computer also happens to be one of the most enduring games of all time.

Introduced in 1984, "Tetris" has sold 50 million licensed copies. New versions of the game are being released by THQ Inc. for next-generation consoles and, of course, the PC.

What gives the game such longevity? "Tetris" is easy to play but difficult to master, taunting players for hours at a time.

Despite changes in the game's graphics, the core game play is essentially identical to the original version created by Alexey Pajitnov to run on a Russian computer on a black and white screen using only text-based shapes. Pajitnov was a scientist with the Computer Center of the Academy of Sciences in Moscow when he began making puzzle games as a way to test new computers.

His first game involved seven shapes built from four square blocks, giving rise to the name "Tetris" that comes from the Greek word tetra, meaning four. The shapes fall from the top of the screen--players rotate them as they fall to fit with the changing grid, below.

Although the game has achieved spectacular sales, Pajitnov did not begin receiving royalties until 1996. Before that, the money went to the Academy, his former employer, and the Soviet Ministry of Commerce. To remedy that, Pajitnov and Henk Rogers established a company, now called Blue Planet Software Inc., to develop and license all "Tetris"-related products.

These days, Pajitnov, 46, is designing puzzle games for Microsoft Corp.

Q: What made "Tetris" such a popular game?

Nobody knows. That's the biggest secret. There are several theories. One is that it appeals to the human need to have order and harmony. You have a random, chaotic situation, and your mission is to create order. The other theory that all your achievements, the blocks you arranged, disappear as soon as they're lined up. So what you have in front of your eyes is all your awful mistakes. It makes you want to fix it all the time. But again, nobody really knows.

Q: What did the original game look like?

Imagine you have black and white screen, no color, no sound. The field is dotted with asterisks. The squares are created by using square brackets. Imagine working with "Tetris" in text mode only, where you're working only with letters and numbers. But if you look at it, you will immediately recognize the game.

Q: How has "Tetris" evolved?

There were several steps in its life. The first was my own design. I designed the game for a very old Russian computer called the Electronic 60. There weren't graphics, so in order to draw a square, I used two square brackets. That was the first version. Then the game was published in the U.S. for the first time in 1988 by Spectrum HoloByte. The big step, however, was with the Game Boy version in 1989. For many years, that was the best possible version of "Tetris." It was very well balanced. The rules included everything I really wanted. The squares were soft-dropped, not hard dropped so you could see the blocks as they fell. That was a pleasant small detail that makes the game more controllable. You'd be amazed how small details are very important to players. The next major improvement came with "The Next Tetris," published three years ago on PC and PlayStation. This version introduced the "cascade" effect, or a chain reaction that occurred like an avalanche when you completed a row. This created an unbelievably good feeling for the player. It's one of the best feelings in gaming you could imagine.

Q: How involved are you with the development of "Tetris" now?

I am still involved in the design. Really, though, there are lots of designers and developers who did many of the improvements in "Tetris." My role is very modest. All the new versions for "Tetris" were designed by Henk Rogers and other Blue Planet Software designers.

Q: How could you possibly improve "Tetris?"

In the newest version that will come out this fall, we have six types of puzzles. Each version is for a different audience. For example, we have a puzzle called "Hotline." This game will be good for beginners and for players with a short fuse--people without patience who want feedback right away. Then there is "Squaretris." That's where you can make other configurations. This version is a little more strategic. People who like to think will really enjoy it. The fact that we have such variety is very good. It will keep up the feeling of novelty.

Q: Are three-dimensional puzzle games necessarily better than two-dimensional games?

I am a professional designer. So all my managers asked me for 3-D. I know 3-D is very fashionable now. It looks very good on the screen. But our brains haven't adapted. People have problems with 3-D. Look at Rubic's Cube. It's three by three by three, like a tick-tack-toe cube. But it's extremely difficult for a regular person. That's why if you want to do a 3-D puzzle, you should be really cautious. The best achievements are in two-and-a-half-dimensional puzzles. Take a regular 2-D puzzle, and put it on an interesting surface. It looks good, but the level of complication of the puzzle remains reasonable. Switching to full 3-D is no problem technically. The main problem is that people won't be able to play them. Basically that's what happened with "Tetris." In 1990, there were two puzzle games, "Blockout," which was a fully 3-D game, and "Welltris," a 2-D game. "Welltris" was one of the top 10 games in 1990. "Blockout" wasn't.

Q: What makes a good puzzle game?

Every game is a very complicated product. For me, it's like a movie. The idea is the most important thing, what kind of mechanics are in the game. The second thing is the balance. It should have rules simple enough to be understandable, but the game should still be challenging. From a designer's point of view, this is the most complicated thing. Graphics and sound have become more and more important. In early years, we didn't hear about this stuff at all, and now it's one of the critical things. The rest is just small professional secrets.

Q: Do you have any advice for a budding game designer?

Just sit down and do it. The good designer comes with his game and says what's wrong with it.

Q: Where do you draw your inspiration?

Different designers have different sources of inspiration. Myself, I usually start with a traditional puzzle. If you come to my office at work, you will see it is full of toys--wire frames, jigsaws, pyramids, and lots of toys with interesting shapes. I work with them all, and I like to solve them.
 

The Anatomy of a Search Engine by Sergey Brin and Lawrence Page

In this paper, we present Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext. Google is designed to crawl and index the Web efficiently and produce much more satisfying search results than existing systems. The prototype with a full text and hyperlink database of at least 24 million pages is available at http://google.stanford.edu/ To engineer a search engine is a challenging task. Search engines index tens to hundreds of millions of web pages involving a comparable number of distinct terms. They answer tens of millions of queries every day. Despite the importance of large-scale search engines on the web, very little academic research has been done on them. Furthermore, due to rapid advance in technology and web proliferation, creating a web search engine today is very different from three years ago. This paper provides an in-depth description of our large-scale web search engine -- the first such detailed public description we know of to date. Apart from the problems of scaling traditional search techniques to data of this magnitude, there are new technical challenges involved with using the additional information present in hypertext to produce better search results. This paper addresses this question of how to build a practical large-scale system which can exploit the additional information present in hypertext. Also we look at the problem of how to effectively deal with uncontrolled hypertext collections where anyone can publish anything they want.  

CommuniGate versus Microsoft ExchangeBy Nicholas Petreley

Last week I got a call from Stalker Software Inc. Stalker makes CommuniGate Pro, a full-featured mail server that supports SMTP, POP3, IMAP4, ACAP, LDAP, and a few other standards acronyms. It includes an extremely well-designed Web-based administration system and Web-based access to email. It includes antispam features, mailing list management, and even personal Web sites for your users.

The reason Stalker called is because, according to the company's own benchmarks, CommuniGate Pro on Linux skunks Microsoft Exchange in terms of performance, based on Exchange benchmarks performed by IBM. The comparison is between Exchange 5.5 SP2 running on Windows NT 4.0 SP4 versus CommuniGate Pro 3.1 running on Red Hat Linux 6.0.

Stalker used the same brand of machine, the IBM Netfinity 3000, but actually used less powerful hardware than IBM to run its tests. Stalker has published the details of its tests at http://www.stalker.com/MailTests/Matching.html, and compares the results to IBM's benchmarks for Microsoft Exchange (see ftp://ftp.pc.ibm.com/pub/special/serverperformance/nf3000_exch_090899.pdf).

LinuxWorld does not endorse either benchmark, but we find them both to be believable. And that isn't only because -- unlike the recent benchmarks by Mindcraft and ZD Labs -- these results tend to support the anecdotal evidence that Linux outperforms Windows NT.

Although the benchmarks themselves are an apples-to-grapes comparison, a cursory examination of the Stalker benchmarks reveals that the company's approach appears to be far more solutions-oriented and realistic than that of Mindcraft in its benchmark several months ago. For example, Mindcraft pummeled servers with requests for static Web pages, which is rarely how a Web server is used today.

Stalker could have contrived a similarly unrealistic test scenario for mail testing, such as confining all its mail traffic to be internal to the server. To its credit, it created a far more realistic test environment. It routed a good portion of its mail outside the server, and even added nice touches, like simulating POP3 users checking their mailbox when no new mail was waiting.

We find the benchmarks particularly interesting in view of the continued rapid growth of Linux in the ISP market. If CommuniGate Pro truly boasts the speed and scalability claimed by the vendor's benchmarks, it is increasingly likely that it might be hosting your next ISP mail account.

All things considered, I recommend you keep an eye on Stalker Software Inc. To help you along, we spoke with Stalker Software Inc. President Vladimir Butenko about the benchmarks, and about Stalker's strategy of competing with Microsoft Exchange.

LinuxWorld: How does the hardware compare between your benchmarks and Microsoft's?

Vladimir Butenko: There is a difference. We both used a Netfinity 3000. They also had RAID 0. We had one plain SCSI drive. Their computer was 550 MHz. Ours was 500 MHz even. They had 768 MB of RAM, while we had just 64 MB. So generally speaking, what we have is just an out-of-the-box Netfinity 3000. What they had was a Netfinity 3000 with a lot of additional things that cost much more.

LinuxWorld: [in disbelief] You had 64 MB of RAM?

Vladimir Butenko: Yeah, that's it. And it didn't use all of it for the test. The server, during a 3 hour period, it needed approximately 25 MB of RAM. And that was it. Because what we actually tested was the type of environment you can now see in the office. Not the type of environment you'll see in a couple of years where most people will switch to Web email. Those things require more memory. But for regular use, CommuniGate is extremely efficient on resources. On this kind of load it didn't need a lot of memory.

The bottom line is [that] we succeeded in submitting 4.5 times more messages. So in all areas, it was 3 to 5 times faster than what they have on that beefed up Netfinity.

On the other hand, they claim 70 percent CPU utilization. In our test we were well below that -- between 29 and 31 percent. We do not think that any server that consumes 70 percent of CPU can be used in a production environment. Any peak load will bring it to its knees. You need to have some space in CPU power.

Also, unlike those guys, we did not use a 100BaseT network. We used 10BaseT, regular 10BaseT. So it's more common to the office [environment]. It emulates the normal kinds of collisions on the net.

If you have a clean connection to the Internet, and [a] very, very steady load, then probably you can tolerate a 70 percent CPU load. But in a real-life environment as we saw in our tests, the load changes from time to time. If you're already at 70 percent CPU, you'll easily run out of steam, which causes a huge degradation of speed.

LinuxWorld: If they had used 10BaseT, wouldn't that have reduced their CPU load?

Vladimir Butenko: They wouldn't be able to submit the high number of messages in that case. And we submitted 4.5 times more! If we had used 100BaseT, we would have been able to submit 7 to 10 times more messages. The problem is the disk. What we saw as a bottleneck was the disk I/O system on this particular machine.

Actually, this is true of all installations of CommuniGate Pro. It's easy on the CPU, and as fast as your disk I/O system allows. If we had used RAID it would have been much faster.

LinuxWorld: Realistically, how important is messaging speed? Does it really matter if it takes 5 or 20 seconds for a message to be delivered?

Vladimir Butenko: That's not the point. Sometimes it's important too -- the time of delivery. But the main thing is that I'm talking about the queue. And not delaying that queue. If the speed of your disk I/O system is not fast enough for the load you put on the server, the queue starts to grow. And if it starts to grow, it will never [stop growing].

If you see that the queue has started to grow, then within a day or a week, you'll definitely end up with a million messages in queue. On average we saw about 50 message in the queue. But these messages should appear and then go. If it starts to grow, it means your disk I/O system is not up to snuff.

LinuxWorld: How big did the Exchange queue grow?

Vladimir Butenko: They claim 1000 messages, but it's hard to compare because the queuing mechanisms are different. The main thing is that the queues in all the steps -- and we hope that Microsoft didn't cheat here -- that the 1000 messages were during the test and not at the end of the test. That means the queue didn't grow. If it is steady for several hours at some certain levels, that's OK. It didn't grow. The most important thing is that the queue does not grow.

We also checked different tasks. They definitely used the proprietary MAPI protocol. We used standard SMTP and POP as most of the mail systems support.

What we have in difference here is that they claim they established all the connections they needed for the test, and then performed those operations with those connections. In our case we couldn't do that, because it wouldn't be real for an SMTP or POP system to keep the connection open all the time.

Also, on big servers, there are many POP sessions where people check their mail but nothing is there. So, during our tests, we had a test load that emulated connections to a mailbox with messages, where it delivered and deleted all messages. But also we had connections to empty mailboxes. This part was completely absent in the Exchange test.

One thing I wanted to mention was that the document for the Exchange benchmark was not clear as to whether messages were sent outside of Exchange. Because in the references, it says "messages sent" means messages transmitted from Exchange to MTA. On the other page it says X number of messages were sent. We don't know if any messages were sent outside the server.

So we don't know if the test included the time needed to send messages outside the server. In the corporate environment, we do not believe that all the communications is within the corporate network within one server. It is completely unclear from their document if the test sent outside the server.

LinuxWorld: What about IMAP testing?

Vladimir Butenko: It's too difficult to compare the operations. We would like to do that because people need some type of benchmarks on IMAP. But in the regular office, it is much easier to compare what Outlook does and what a regular POP client does.

For example, with IMAP, a lot of people are interested in copying between mailboxes and things like that. These are things that were not reflected in that particular Microsoft Exchange test.

LinuxWorld: How do you compare feature-wise? In order to convince customers to consider CommuniGate Pro as an Exchange replacement, don't you need to have the calendar features and the ability to perform as a drop-in replacement for Exchange using Outlook?

Vladimir Butenko: The only thing we are lacking is full support for the proprietary calendaring. CommuniGate Pro already supports IMAP-based calendaring with Outlook. And we hope that by the end of this year we'll have complete support. And also, Microsoft is moving Outlook calendaring to a standard.

In other areas, CommuniGate Pro offers much more functionality than Exchange. It has the same security features as Exchange in terms of secure connections [and] Web interface.

If we look at antispam features, for example, Microsoft Exchange has only the basic set. We have a complete set of antispam features. If we look at IMAP servers, Microsoft has the implementation of the basic IMAP function. We have the complete implementation of IMAP, including ACL, shared mailboxes -- this applies to everything.

LinuxWorld: Have you looked at HP OpenMail for Linux yet?

Vladimir Butenko: We know that it exists. It looks like a direct port from their old system to Linux.

LinuxWorld: I found it more difficult to administer than CommuniGate Pro. But we're talking about competing with Microsoft Exchange now, and one thing HP OpenMail has is that MAPI plugin -- so that as far as Outlook users are concerned, they don't know that they are not talking to an Exchange server.

Vladimir Butenko: We don't think we'll do the same thing. My crystal ball says Microsoft will move to a calendaring standard very soon. Calendaring is the only feature that people use MAPI for.

LinuxWorld: So you're saying you'll end up in the same place as HP Open Mail but you're taking a different approach?

Vladimir Butenko: No. The calendaring protocol is already standardized. It was standardized almost a year ago. November, I believe. Both Netscape and Microsoft committed to move their clients from proprietary protocols to that protocol. They are all moving toward this standard -- not fast, but they're moving. They will all use that standard protocol which is now only supported by Sun calendaring server.

In addition, Outlook can support calendaring features over IMAP. But Microsoft uses something proprietary which we haven't discovered yet. So as long as you keep connected to the server, the calendaring works. But if you disconnect and someone else connects to the same mailbox, something happens so you can't use it.

LinuxWorld: It seems like a bold assumption to say that Microsoft is actually going to fully support an open standard. Their pattern is to keep at least something in there proprietary to lock you into a Microsoft product. So how do you deal with it?

Vladimir Butenko: I asked the lab to check what's going on. We can see at least what Outlook attempts to do so we know where the problem is. I believe that they use a couple of nonstandard commands to keep things synchronized. If we see those commands, we'll put them in and support them.

We still think that major manufacturers, including Microsoft, will migrate to standards-based calendaring. What they use now is not standard. They just use IMAP as a calendar store. But we'll be doing both: we'll be concentrating on the standards-based calendaring, and we'll see whatever nonstandard things we have to do with IMAP in order to support calendaring with an Outlook client.

LinuxWorld: If you had to draw a pie chart of where your sales are, which platforms you sell into, how would it look?

Vladimir Butenko: On the lower end, which means the corporate or enterprise license from 50 to 200 accounts, or even to 1,000 accounts, most of the systems we see are NT, Linux, or Mac OS. On the high-end, which is something less than 100,000 accounts, we have 80 percent on Solaris, 15 percent FreeBSD, and maybe 5 percent Linux. We have the exotic systems like AIX, too, but they're usually in the middle.

LinuxWorld: What's your high end on accounts per server?

Vladimir Butenko: If it is not a cluster, it is usually 100,000 or below. We do not recommend running more than 150,000 or 200,000 accounts on one server. Many sites have between 30,000 and 70,000 accounts, and most are below 150,000. If you go beyond that, you need a cluster.

At ISP Con, we will show static clustering. I don't believe we'll be able to show everything there, like dynamic clustering. The static clustering is designed for a site with a size of up to 1 million accounts. And dynamic clustering is needed for a live site where fault tolerance is very important, because dynamic clustering allows you to switch to any server in the cluster at any moment without any reconfiguration. All servers are dynamically recognized. If one of the servers dies, it redistributes the load automatically.

LinuxWorld: Thank you for having taken the time to talk with us.

Cyrillic fonts Russian homophonic keyboard; Russify browsers; Mail and News Russification -- an excellent page for Windows users

WebCopier Multifunctional Offline Browser by Maxim Klimov

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019