Mystery Google Device Appears in Small-Town Iowa (2 of 2)

By Cade Metz

Networking ports on the Pluto Switch were unlike any the forum had seen.

‘I Know Who Made This … Just by Looking at It’

When we showed the photos to Andrew Feldman — the founder of SeaMicro, a new-age server outfit now owned by chip designer AMD — he said he knows the engineer who built it. “I know who made this … just by looking at it,” he told us over the phone. “I know the engineer. I’m looking at it, and I know an engineer’s work.”

He wouldn’t say who the engineer was, and he didn’t link the switch back to Google. But Feldman has a history with Google. He once worked for Force10 Networks, a company he says sold hundreds of millions of dollars in networking equipment to Google — before Google started designing its own gear.

When we showed the photos to Google, the company said they were old. “I don’t think we have much to say on this front,” a Google spokesperson told us. “These forum posts are from over 7 months ago.”

Nowadays, Google designs most of the hardware that drives its web services, including servers and the data center facilities themselves, and it sees this custom hardware work is a competitive advantage over all the other big-name websites struggling to instantly accommodate requests from millions of users across the globe. That’s why it treats the hardware as a trade secret.

Earlier this year, after years of rumors, Google revealed that it designs specialized networking hardware for moving data between its data centers — hardware that uses a new protocol for managing networking gear called OpenFlow — but the company provided few details about the hardware itself. And it’s still tight-lipped about the networking hardware used to move data inside its computing facilities. As recently as June, Urs Hölzle — the man who oversees the Googlenet — declined to discuss the matter.

But the information posted to the forum indicates that the Pluto Switch is at least akin to devices Google uses inside its data centers. An “edge switch” is a device that connects servers inside a data center to a larger network. These are also called “top-of-rack” switches, as they typically sit at the top of a rack of servers.

Ex-Google engineer J.R. Rivers — who now runs a networking outfit called Cumulus Networks — says this appears to be an older switch, something that has been used for about three years or so. And he questions whether Google would put label like “Pluto Switch” on its custom-designed gear. But in all likelihood, this is a device that belongs to Google, and as described in the forum, it’s representative of the company’s overall hardware philosophy.

With its custom hardware, Google aims to improve the operation of its data centers, but it also seeks to reduce costs. Because it operates at such an enormous scale — the company now runs its services across as many as three dozen data centers across the globe — it can save vast amounts of money by reducing power consumption and stripping hardware to its bare essentials.

According to Rivers, the Pluto Switch is an example of this sort of streamlining. He points out that the “physical layer” that transports data across the device is a stripped-down version of what you’d see in a typical switch. It still uses optical fiber lines to move data, but it relies on a much less expensive collection of hardware.

A look inside the Pluto Switch, where a Broadcom “Scorpion” chip drives the device’s 10 gigabit connections

Those networking ports on the Pluto had many people on the forum scratching their heads. “That must have come from outer space (as the name also implies),” said one poster on the forum, responding to the photos. But Andrew Feldman says they’re a way of cutting costs. “This is a different type of connector, because connectors are extremely expensive,” he told us. “They’re designed to take a modified cable. My guess is they’re making the cable as well.”

Where the World Is Going

The Pluto Switch was built for a network that can theoretically handle 10 gigabits of data a second, a big step up from the 1 gigabit networks that still dominate the data center landscape. According to Rivers, it offers 20 ports for 10-gigabit speeds and four others for 1 gigabit.

Other information posted to the forum, Rivers says, indicates that the switches use a central 10-gigabit chip known as Scorpion, a processor manufactured by Broadcom. Broadcom now offers a more advanced 10-gigabit chip known as the Trident.

Google was at the forefront of the move to 10-gigabit Ethernet networks, and the technology is now starting to spread across the industry. Facebook just recently moved to 10-gigabit Ethernet inside its new data center in western North Carolina.

The question is whether the Facebooks and the Amazons will also follow Google’s efforts to significantly cut the cost of its networking gear. But they’re already moving in that direction.

In designing its own gear, Google also reduced costs by removing the middleman. Rather than purchase switches from big-name hardware sellers such as Cisco and HP, it took its custom designs to manufacturers in Asia — the same manufacturers who build hardware for the Ciscos and the HPs. In recent months, other web giants have followed suit, looking to buy their own dirt-cheap gear directly from Asian manufacturers.

They’ve yet to go as far as Google. Odds are, they’re just buying commodity gear from Asia rather than coming to manufacturers with their own designs. But this may change. They too need ways of cutting big costs across their data center networks — and Facebook is already designing all sorts of other hardware for the data center.

The other web giants may not have access to a Pluto Switch. But you can bet they’re looking at something an awful lot like it.

Additional reporting by Robert McMillan

Correction: This story has been updated to correctly place Council Bluffs in Iowa.

<< Back

Source: Wired 2