Audio file

KTLO Bob Wagner.mp3

Transcript

00:00:01 TODD

What I like about being in this line of business and distribution is that I get to be involved in a lot of the behind the scenes work that goes into much of what runs the infrastructure of our world.

00:00:12 TODD

Distribution in the companies we work with help supply the boring stuff that makes the cool pretty stuff run. In this case, we're all aware of the blinking, high power, cool computer processors that power and crunch the data that for the fun, functional, powerful stuff on our phones, our desktops running vim, and even the context.

00:00:32 TODD

Equipment on the job.

00:00:34 TODD

Those blinking light processors and servers? Well, they're just part of the story, with AI computing increasing daily, we're seeing heating and power consumption problem. Engineers continue to improve the design of the processors to be more efficient. At the same time. And there are hundreds of companies creating products that bring power.

00:00:53 TODD

Cooling data pipelines, connections etcetera, etcetera to those servers they are working to make sure those servers run cooler and more effectively.

00:01:04 TODD

Today I'm going to be speaking with someone from one of those companies supporting that infrastructure and that's pinned to it. Bob Wagner is pinned to it, senior development manager for data centers, and he and I will be breaking down the heating and power consumption dilemma. We will also discuss the ways the infrastructure can dissipate the heat and support the work of the processors.

00:01:23 TODD

Before we get into the show, let's get to know our guest. I asked him, who is Bob Wagner?

00:01:31 BOB

Kid from a medium sized town in the Midwest and was very good at math. So he went into engineering, found out his first engineering job. He was too energetic to sit at a table very long, so decided to go back to grad school and and get into business and has been doing that.

00:01:50 BOB

Their sense, and is, is lucky enough to be one of those people to fill in to exactly what he wanted to do, create new products, develop new things and stay close to technology.

00:02:01 TODD

Welcome to keeping the lights on. I'm your host, Todd Reed, and on this podcast I connect with the owners and pros who design, build and maintain our electrical, communications and industrial world to explore the best ways forward before we get going. Let's hear a word from our sponsor.

00:02:16 TODD

AI is transforming industries, and it all starts with the infrastructure behind it. Whether you're expanding your data center or upgrading for next Gen. performance panduit, as you covered now, panduit's trusted solutions from high density fiber to intelligent Power Distribution are engineered to handle the speed and scale AI.

00:02:35 TODD

Commands and working with their distribution network, you'll get what you need where.

00:02:40 TODD

And when you need.

00:02:40 TODD

It see the links in the show notes to visit graybar.com to explore panduit's AI Ready Infrastructure solutions today. OK, let's get into the show. Well, Bob, as we talked about before, you know, we like to start out each episode with the meals that bring us together.

00:02:56 TODD

So Bob, if I were I or one of the listeners viewers or to visit you in your hometown, where might you take us and why would you choose this?

00:03:03 TODD

Place.

00:03:04 BOB

Oh, that's an easy one because I live in Naperville, IL and we have a lot of good restaurants. But Mason Sabika, it's a tapas restaurant that is cannot miss it during the summer. They've got this huge patio, big oak trees overhanging the food's great. The sangria is great. It's just a, you know, we have.

00:03:24 BOB

The Schwartz summers and then in the Chicago area to get outside. Everybody wants to do it and that is the best spot nearby. It's really good.

00:03:32 BOB

Food.

00:03:33 TODD

Awesome. When you mentioned a a drink, but do you have a favorite food item that you like?

00:03:37 TODD

Love to get there.

00:03:39 BOB

Well, there's a couple of them. The Pincho de Solomeo is basically skewers of beef with horseradish and some onions. And then the other really good one is dates wrapped with bacon. Obviously putting bacon on anything is good, but theirs are fantastic.

00:03:56 TODD

Yeah, that sounds great. The only problem about asking these questions at 10 AM is that I'm like, oh, great. Now I'm hungry and coming cut this short naperville's not too far from here, relatively speaking.

00:04:08

All right.

00:04:08 TODD

Well, yeah. I'm in Saint Louis. So yeah, I can get there, you know.

00:04:13 TODD

Dinner time. Hey, so that restaurant you know, very likely there are some process in that restaurant that they're utilizing AI. I mean, who isn't using it somehow and we're probably using it when we don't even know we're using it frankly. And so AI computing I mentioned in the in the intro and it certainly I could it goes unsaid.

00:04:33 TODD

That's, you know, the computer used for that is growing and the power usage, but there's a lot of challenges with the heat generation, you know, and power consumption because of AI computing. So what are some of those new challenges that are popping up day by day as AI computing power continues to grow?

00:04:50 BOB

Yeah, you're seeing the power consumption needs shooting off the charts. We were just looking at this back in 2014. The Uptime Institute did a survey and that surveys thought we'd be at 40 kilowatts per rack, which is actually where AI has got us now.

00:05:10 BOB

But the odd thing is when you look at a traditional data center, they're less than 10 kilowatts, so jumping from 10 to 40, and now the newest NVIDIA system is over 100, is about the NVL. 72 systems are 120 to 140 kilowatts.

00:05:27 BOB

That is, that's that's a gigantic increase. And then the road map that our friends at NVIDIA have developed in just a couple of years are going to be at 600 kilowatts per rack, not not building but rack and so 10 to 600. That's a 60X increase.

00:05:40 TODD

That's great.

00:05:45 BOB

So what? I'm only telling people is if you have a data.

00:05:49 BOB

The center. It may be obsolete and maybe they just don't have the power coming into the building. They don't have the infrastructure inside the building to run power like that to one rack. They don't have the cooling, it's putting a ton of stress on data centers that were built just even a couple years ago. So it's it's a big issue and then.

00:06:10 BOB

Of course, from there it moves into the cooling. OK, so now maybe I've got the power and you know, a lot of these buildings are 208 volts inside. It really isn't going to cut it, right. You just don't have enough. You have to be moving into 415 volts. So even if you get the power now, you got to talk about cool.

00:06:19

Hmm.

00:06:28 BOB

And when you get to be 50 kilowatts or more.

00:06:32 BOB

You can't blow air. It just doesn't do it.

00:06:34 BOB

It's not strong.

00:06:34 BOB

Enough. So now you got to look at liquid cooling, and there's several different styles of liquid cooling to look at, and every one of those comes with complications. In fact, I I heard a guy from digital reality kind of joking, but not joking. I'm going to have to hire a lot of plumbers because it's either water or some other liquid, and it's going to be.

00:06:52

Yeah.

00:06:54 BOB

Very prevalent in AI systems going forward.

00:06:58 TODD

How does all that demand for higher computing performance affect the space use in data centers?

00:07:04 BOB

There's a lot of, you know, questions about that, because some people, you know, maybe a little light heartedly would say I'm going to have this great data center and I have 6 racks and there'll be a bunch of empty space because those racks are operating, you know, 120 to 600 kilowatts each. I just don't have enough power coming in.

00:07:25 BOB

But no, I don't. I don't really think that's going to be that way. When we talk to the biggest of the big you know, the hyper scalers, Microsoft, Amazon, Google, etcetera.

00:07:33 BOB

They're still driving density, they want smaller and smaller fiber connectors. They want to maximize that density. But I will tell you I've also seen the other way is people using bigger cabinets and not worried about using bigger cabinets which take up more space because there's more fiber than ever in these data centers and they don't.

00:07:55 BOB

Want to have this congested area where they just can't get the fiber to run and they want to spread things out a bit and we're talking about air cooling and blowing air, which you still will do even if you're using liquid cooling, you're still going to be blowing some air.

00:08:10 BOB

Moving the rows farther apart is certainly a help as well, so it's it's it. I don't know, it hasn't really washed out yet. I mean they they still want the density in the rack.

00:08:21 BOB

And they want things as close as possible, like server to server, GPU to GPU as close as possible, but they also I think they'll run out of energy and power coming into the building and for thermal properties they'll probably be spreading things apart. Rows will be farther apart.

00:08:37

Hmm.

00:08:39 TODD

So can you walk us a little bit through how cabling and pathways which are affected by this shift to two racks designed for AI computing?

00:08:50 BOB

Yeah, the first thing it is to think about is that most traditional servers use two fiber which is a duplex LC, so they only have two fibers coming in.

00:09:01 BOB

Now with GPUs, they're all going to use 8 fibers or potentially more in the future coming in. So they're using parallel processing and that that's an MPO. So that that alone has quadrupled the amount of fiber coming into the server racks.

00:09:18 BOB

And then the architectures which is real optimized, which is it's a, it's a cloth, it's it's, it's a, it's a different variation of spine leaf, which a lot of people are familiar with, but it's very intensive. And what they're trying to do is have every GPU talk to every other GPU, know how matter, how big that cluster is.

00:09:38 BOB

In the least amount of switch stops every time you go through a.

00:09:42 BOB

Which?

00:09:43 BOB

You get latency and they want lowest latency and that just means they're they're talking is like 1 seamless brain. So if you can shrink all that and make them talk as one seamless brain low latency, Now you've really got a good AI computer. But to do that it becomes kind of tricky. You basically have to start shoving.

00:10:03 BOB

In in in sending and routing fibers in many many different directions. So there's estimates that the the fiber usage is going to be 8 times more than the traditional data center.

00:10:16 BOB

So if you're trying to retrofit an existing data center, likely you may have to pull out all your pathways that are above the racks because you just won't have enough space in there. This fiber takes up a lot.

00:10:29 BOB

Of.

00:10:30 BOB

Room we're working with customers right now to put, you know, take a bunch of individual fiber.

00:10:37 BOB

Assemblies and shrink them into one single jacketed trunk because that dramatically reduces the space is even in the rack, especially when you're talking about like an NDL 72 system.

00:10:50 BOB

All the fiber and even the copper for the out of band really gets into a congestion point at the top of those racks. And by shrinking those things down by consolidating multiple cables into one jacketed cable, you can really help yourself and make it way more manageable.

00:11:07 TODD

So what are the operational risks if heat and space challenges aren't address?

00:11:12 BOB

You don't have an option. To be honest, you really need to plan ahead. In fact, one of the main things when we talk to customers or we do any training to people, you got to plan ahead. It's not like just another data rate increase going from 100 gig to 400 gig.

00:11:15 BOB

You.

00:11:32 BOB

That that extra power you really have to have a data center that can handle it. And if you're really going to start putting in some of these hiring systems like a a B200B300 or an NVL 72, it becomes mandatory to have liquid cooling and you just don't.

00:11:52 BOB

Suddenly have liquid cooling. You have to have plumbing. You have to have a strategy. You really have to plan this out.

00:11:58 BOB

So many people are saying I want an AI, I gotta get an AI, I'm going to buy myself some GPUs. And it's funny. I hear this every day and then they don't know really what to do with them. They don't even know what their application will end up being. And they're having a hard time figuring out how do I put all this together?

00:12:15 BOB

You got to plan ahead. You got to think about the power 1st and that'll lead you into the thermal and the cooling and then you've really got to lay out enough pathways and and and permanent infrastructure to handle everything.

00:12:29 BOB

You're going to put in.

00:12:31 TODD

So are there challenges unique to retrofitting existing data centers compared to building new ones for AI?

00:12:37 BOB

Absolutely. A lot of people are saying, oh, I'm just going to go build a new one and that's why we're seeing this giant boom. I mean, I was just reading the other day. I think it was from JL and.

00:12:51 BOB

And CBRE that, you know, they're the experts in this, saying that data center.

00:12:56 BOB

Open capacity is like an all time low. I think it's like they said.

00:13:00 BOB

2:00 to 4:00.

00:13:01 BOB

Percent, which means and. And I love the one of their quotes. Was something about, you know, build it and they will.

00:13:09 BOB

And you know, and we've all been joking about, don't do that because, you know, you really have to do more homework. They're talking about.

00:13:15 BOB

Order ahead before they even build it. I mean, before these buildings are even going up, they've already signed someone to be inside of.

00:13:23 BOB

It.

00:13:24 TODD

Oh wow. OK.

00:13:24 BOB

So people have taken the pre leasing years in advance just because they know their existing buildings.

00:13:31 BOB

Or maybe not going to work very well with AI, but let's let's let's fall back. There's a ton of buildings out there that are very functional today, and you know, as we move into more and more AI, they are going to have to be retrofitted. The first thing, as always, power. What do you have coming into that building? Can you run 4/15?

00:13:52 BOB

Volts inside the building because if you can't.

00:13:56 BOB

It's going to limit you, right? You just there's just nothing you can do and getting a new power line from the grid, from your utility, get in line, that's going to be years in, in advance. So that's not going to be a quick runner. So that's your first thing to think about. And then from there, if you can get 415 or even if you're going to run a smaller AI system and run it.

00:13:58

Hmm.

00:14:16 BOB

208 volts inside. Then you can you can do that, but it's going to limit your scope. But you know those are the main things they have to look at and then do you have a raised floor is you have a raise.

00:14:29 BOB

Floor. Not many people do in North America, but it's very popular in Europe. Then you can put all your liquid cooling underneath that raised floor business. Generally you have a lot of room. You were blowing air. Now you'll just pump liquids. That's fantastic. But if you don't have it and most North American don't now it's got to be.

00:14:34

MHM.

00:14:48 BOB

Above.

00:14:49 BOB

You need the plumbing put in place and those connection points are very expensive is they're dripless because you don't want water falling on your electronics because it's all it's going to sort it out and burn it out and at, you know, $200,000 a server you you really can't afford that.

00:14:59 TODD

No.

00:15:08 TODD

Well, you know, power is what we talk about a lot here because that's what we you know your company, our company, you know the people we supply they that's what we deal with most of the time. But what's interesting to me and of course and there's power challenges even with like you know just.

00:15:22 TODD

Charging for cars and then now you've got data centers coming in. Who have, you know, they're hungry too, for power.

00:15:27 TODD

So that's another challenge, but what's interesting is the whole liquid challenge and water and I, you know, I'm not sure where we stand on as far as like water demands in areas where those you know centers are being put in and if that's pushing those infrastructure as well, you know the the water usage and all that cuz.

00:15:44 TODD

You know, I know some of it's a chemical liquid, but I think there's some water usage from local, you know, infrastructure and utilities, I believe so.

00:15:53 BOB

Yeah, a good question. So I have several friends in Phoenix, but you know, you wonder how long can they actually sustain all that it? But there is, you don't have to always use water. I just reading an article just the other day where a glycol or basically you know coolant systems, they come with their own problems.

00:16:13 BOB

Because you just don't like open up a drain and pour that down in because they're not good for the environment, right? That that would be against the law.

00:16:18 TODD

Well, right.

00:16:21 BOB

So you have to you have to manage that, but you can get away with water by doing that. And I think you're going to be forced to. I mean, I sit in Chicago next to the world's greatest supply of freshwater and lake MI etcetera. But it's a scarce resource and you know it's in I think electricity is also.

00:16:41 BOB

Starting to become a scarce resource, the utility companies can't keep up. You are seeing all of the big data centers that they're putting in place. They're with the good thing is they're going with a lot of renewables like wind, solar.

00:16:56 BOB

But they're having to be very, very creative. You're seeing people take leases on existing nuclear power plants like Microsoft took a lease for, I forget how many years on Three Mile Island nuclear is back in vogue. In fact, there's these things called ASMR small modular nuclear reactors.

00:17:16 BOB

Which are maybe the size of a house, and those things are going to go right next door to these these data centers, and you're going to see the data center guys become.

00:17:27 BOB

You know, de facto utility companies, they're going to have to supply their own power because utility companies are just not going to be able to get the right away, to run new power lines to all these buildings. So. And the other thing is you can go. I hear a lot about hydrogen fuel, a little bit farther out there.

00:17:45 BOB

But it's it's going to be anything and and everything is on the table for how do I generate the power?

00:17:52 BOB

And then the location these big training systems that are scraping the entire Internet to create your chat GB, two large language model type systems, they're not going to be sitting next to a big building visit those. I mean, excuse me, the big cities because those cities don't have the power to give.

00:18:11 BOB

They're going to be out somewhere where power is plentiful, inexpensive, and so it's going to be, you know, and and and you don't need to have the training system.

00:18:22 BOB

Generates and have low latency right to the customer. That's where the inferencing data centers are. You can have a training system in the middle of Wyoming, not near any major population area because they've got cheap land, cheap and plentiful power, and you know, places like that are going to become very popular.

00:18:42 TODD

That's why I had John Bob anything about the scraping and the learning systems. That's pretty cool.

00:18:47 BOB

Yeah, there's there's a difference because there's a training data center and inference, and so the the training.

00:18:50 TODD

Training, yeah.

00:18:54 BOB

It's going to run and and really use up a ton of power. It's got, it's got way more electronic servers etcetera, but it it's not on demand, it's training and then it gives it to the inferencing data center using kind of the same equipment, but it's a smaller system, which then when you and I hit a search.

00:19:14 BOB

That's what we're talking to, is the inferencing data center. Now that one has got a difference because it's going to be running 100% of the time, constantly being hit where the training is.

00:19:23 BOB

Going to do some.

00:19:24 BOB

Massive, complex calculations and scraping and then it may go.

00:19:29 BOB

Dormant for a little bit.

00:19:30 TODD

You want to interrupt the show real quick and ask you do you want to keep up with the power demands and performance of AI? Well, start with your infrastructure pan, do it scalable, efficient solutions are engineered to support today's most powerful data center environments.

00:19:45 TODD

Think smarter cable management, faster connectivity, and modular designs to grow with your data center, all available through Graybar.

00:19:52 TODD

Now Panduit is powering the future of AI. Click on the links in the show notes to visit graybar.com today and see how your data center can keep up with tomorrow.

00:20:02 TODD

Let's get back into the show.

00:20:05 TODD

All right. Well, we've talked about a lot of the challenges being faced, lots of lots of heat, lots of power. So let's start to talk a little bit about the some solutions, some ideas about what's, you know, working and where people are going to solve this, you know, heating and heating, cooling, power challenges that are ahead of us. So let's start with how can a data center optimize?

00:20:26 TODD

Power Distribution to support these higher density rack.

00:20:30 BOB

Sure. So today's server cabinets will normally have two PDU's that's just a Power Distribution unit. If you're not familiar with that, it has a lot of outlets, the servers will all plug into that. The servers have upwards of 6 power supplies.

00:20:50 BOB

Each and so you're always shooting for redundancy. So you have a PDU on the left side of the rack and a PDU on the right side, and you'll have half of the power supply is connected to the left, half to the.

00:21:04 BOB

Right. Meaning, if you ever lose power on one side or something gets cut on one side, the other side will pick it up and the server won't go down, which is obviously hugely important. So that normally the normal traditional takes 2P to use one left one right.

00:21:20 BOB

When you start adding the power consumption needed by these, these high end GPU servers.

00:21:28 BOB

Now you're starting to run out.

00:21:29 BOB

The power and so now you start to say, oh, I need 4 and in some cases when you do the math I need 6 PDU's per rack. So it's three per side and some of these are Even so complicated that the server cannot be redundantly kept up. So if you lose the left side.

00:21:50 BOB

The whole thing goes down South. You really need to do your homework ahead of.

00:21:54 BOB

It is not advantageous to try to put 6 pdus in a rack. Takes up a ton of room and then if if if you didn't know a lot of these AI servers are much deeper and the racks themselves haven't really gotten that much deeper. So your room in the back where these PDU's go and where now if you're going to use liquid cooling the liquid cooling.

00:22:15 BOB

Because it's getting tight back there and we're talking about congestion, I've seen a lot of the, the, the folks, the servers moving, their connectivity, all the fiber connectivity to the front.

00:22:27 BOB

Dell was one of the first people to do that. I think that's really, really intelligent. Get that, you know, cabling that can be damaged away from where the liquid cooling is, where away from these power cords are very smart. But getting back to the power you you want to have, hopefully you can only get away with four PDU's and then you need to talk about.

00:22:47 BOB

What are my needs? How much intelligence do I have?

00:22:50 BOB

If you can go from a basic PDU that doesn't do much to one that monitors the power at an outlet level and can do a lot of management, you can even synchronize these together with sensors like a lot of people have locks and you have to badge your your you know into the cabinet.

00:23:11 BOB

He even be able to open it up because a lot of expensive equipment people don't want just anybody.

00:23:16 BOB

In there you can actually cycle that through the PDU's back to the control system, so they're not just power units, they're actually information connecting systems that works with your control system back at the the network office. So you have a lot of different.

00:23:37 BOB

Abilities to make and add intelligence to your entire system using these pdus.

00:23:43 BOB

Now when you get into these NVL 70 twos which are over 100 kilowatts per rack.

00:23:50 BOB

NVIDIA and several other guys have decided, hey, we just can't have that many PD's back there. Let's switch it all out and and and use a power shelf and a bus bar. So the OCP, which is open compute, which has been developed by meta many, many years ago, it's an open forum.

00:24:12 BOB

Has come up with a bus bar and that's just basically a big copper.

00:24:17 BOB

Rail, where all of the servers will plug into, and you're at 48 volts DC directly into the server. The nice thing about that you're not going from AC to DC, you're just going GC directly in and that it makes it more efficient and being able to slide things in is also a.

00:24:37 BOB

Kind of a nice way to go. It does complicate things a little bit, but it's probably the way of future for things are going to be over 100 kilowatts.

00:24:46 BOB

And those power shells I mentioned kind of take this place of the PDU by adding the intelligence, but but for the most part, they're really just taking it and making it DC. They're really just converting AC/DC that.

00:25:00 BOB

Sort of thing.

00:25:01 BOB

So that's where you're you're you're looking for the future?

00:25:04 TODD

So let's start out with the cooling technologies that are emerging as best practices for managing the heat generated by AI workloads.

00:25:14 BOB

Yeah, you have your options and it all depends on where you're at. As far as the kilowatts you're going to use per rack, and and there's some obviously debate. So take what I'm going to say for just as a a rough estimate, around 50 kilowatts, give or take, is when you really run out of being able to.

00:25:34 BOB

Use air containment and blowing air without any other additional factor to it. So at 50 kilowatts per rack, a lot of times you're going to want to move into a rear door heat.

00:25:49 BOB

Danger and basically think of that as the refrigerator you have in your house. It's basically pulling in water. It's it's cooling it off and then blowing air through that. The the rear door heat exchanger. You can take off and take or. Excuse me, you can take an existing cabinet, take off the rear door.

00:26:10 BOB

And then apply this rear door heat exchanger onto it and it's in the same footprint. It's obviously thicker than a typical door, but you should easily have room and now you got a refrigeration unit. So as this hot air come.

00:26:23 BOB

Out of the servers and switches it it goes through the regular heat exchanger and it's cooled so that going back out your ambient air is is is going to be a lot lower temperature and so it it definitely keeps that the overall system cooler. So you can get those to run.

00:26:44 BOB

Maybe up to about 70 kilowatts per rack and.

00:26:49 BOB

That's about 75 is when you really need to start talking about true liquid cooling. The rear door heat exchanger has water going into it, much like your refrigerator does, but true liquid cooling you got a couple different options. The one that's I think everybody sure is going to be the most prevalent is directed.

00:27:09 BOB

Chip liquid cooling.

00:27:11 BOB

Your your chips, your GPUs or CPUs, they all have these big heat sinks which are just nothing but fins that are very thin and you blow air over it. And much like an elephant has ears. And that's how they regulate their own temperature because it's thin and has a lot of surface area. That's how a heat sink works well.

00:27:30 BOB

You can take those monstrously big heat sinks.

00:27:33 BOB

Out and now you just replace it by maybe an inch at most cold plate and the cold plate has cold water coming in. It circulates around. There's a lot of different technologies, but basically it absorbs all that heat coming out of the GPU or CPU. The water heats up.

00:27:53 BOB

And then it shoves it back out and it it dissipates out, and then it's cooled off at a in a tank outside the building.

00:28:01 BOB

That's way more efficient, because why instead of just blowing air everywhere it's blowing or it's cooling the chip itself, it's right there on top of it. And that's very, very effective. And I think that's going to be very popular, the one that is even more efficient and and I should say that pumps.

00:28:21 BOB

With liquid, use up way less energy than fans pushing air, but the one that is the most efficient is the immersion cooling, and that's where you literally take your server.

00:28:26

Hmm.

00:28:34 BOB

And you drop it into a tank of fluid and it's not it's water. It's a dielectric fluid and that works. The whole system is going to be cooled. It's expensive. It's complex. It really changes your architecture. So a lot of people are staying away from that. I mean, the supercomputing systems.

00:28:37 TODD

Oh wow.

00:28:54 BOB

Cray has been doing that for over 25 years. Kind of went away from.

00:29:00 BOB

I think maybe the government labs, the true supercomputers will will use it, but I don't think a lot of the normal data center companies are going to be going that way. Maybe crypto mining at the most. I think the direct the chip is a little bit easier, less fundamentally different architecture and you'll see that be the primary.

00:29:20 BOB

Way to cool things in the future.

00:29:23 TODD

So Bob, can you touch, spend a little bit of time on you know what structured cabling solutions can do to help maintain air flow and manage dense fiber requirements?

00:29:32 BOB

Sure, there's a couple different ways to do your cabling. You can do the point to point which is what a lot of the supercomputers and and previously were doing and which you know, some of these high end systems are still doing. And that just means exactly what it says. Cabling directly from the server to the switch, nothing in between.

00:29:52 BOB

That's going to be the least expensive and it's really good if you have a small AI system.

00:29:58 BOB

But as you start to scale out and you have multiple rows of multiple server racks, it becomes more difficult because now you have those cables going in many different directions and port mapping becomes very, very important. And when you have these cables going around and something gets damaged or something's not put in the right place.

00:30:18 BOB

Debugging it and point to point camlink. It's really tough.

00:30:22 BOB

So what a lot of people have been doing for many, many, many years is structured cabling and what you end up doing there is you have demarcation points. So instead of going directly server to switch or switch to switch, you go server to a patch panel which is just a an adapter or a coupler and then you have another cable.

00:30:43 BOB

To another patch panel and then down to the actual other side of the electronics. So what that allows you to do is before the server racks and switch racks have ever been installed, you can hang your pathways which hang above all these racks.

00:31:00 BOB

You can put in your long cables into those pathways. You can connect them into patch panels, test it, make sure everything is good, so that when you bring in and this is mostly an AI racking and rolling these these cabinets and you slide them under those patch panels up above.

00:31:20 BOB

Of and then all you have to do is a quick patch cord or jumper from the server up to that patch panel, and then if something doesn't go right or there's some mistake, you're just dealing with a maybe a 3M at most patch cord. Very easy to fix and you don't have to go digging into the pathways up above so it simplifies.

00:31:41 BOB

Troubleshooting.

00:31:43 BOB

It actually speeds up dramatically speeds up the install because all the cabling is put in place that you don't want to touch, and then you're just dealing with the short patch chords.

00:31:53 BOB

And then you can also put trunks with structure. Cabling allows you to, like I was saying earlier, consolidate a lot of those individual fibers into single trunks, trunks as high as like 864 fibers, 303,456 fibers and that takes up a lot less room in the pathways.

00:32:14 BOB

So there's the the the structure came, one gives you a lot of advantages debugging.

00:32:19 BOB

Removes the slack and if you do have slack and you've overestimated how long your cables are, it gives you room to coil that stack up into so.

00:32:29 BOB

That's really a big help when you're trying to future proof and think about adding on to your system and just making it go up and be running as quickly as possible, right? Right. As soon as you get those racks put in place.

00:32:43 TODD

OK, I want to talk about a very super tactical thing, but in my intro I talked about the all the things that go into running the infrastructure of our world and it's this specific case AI computing and it goes down to every little detail and every little detail feeds into the bigger picture. You know, those fancy pretty light servers and all that stuff that everyone knows and doesn't know about, all the stuff that.

00:33:04 TODD

We're talking about. They don't see it.

00:33:06 TODD

So I'm curious. Let's get down even to the connector. Is there something connectors can do to make a difference in this space and installation challenges we?

00:33:16 BOB

Yeah, there's several different products have been coming out over the last couple of years. The DS FF, which is just stands for very small form factor is doing exactly what you're just asking for. It's shrinking. You have one that shrinks down that duplex LC to 1/3 of the size and now you've got one.

00:33:36 BOB

AI, which shrinks down the MPO's down into one third the size the most popular ones are the Cinco SN Mt and the US Connect M.

00:33:51 BOB

And see both of those are primarily right now 16 fibers, so they're about 1/3 the size of a MPO that can host 12 fibers. So now you're getting more fibers and less space. This transceivers have not caught up with that yet, so you still need an MPO when you go into the server, into the switch.

00:34:13 BOB

But when you go into your patch panel.

00:34:15 BOB

Imagine now you can triple, quadruple, you know and and put up upwards of 3456 fibers in a patch panel. Not sure everybody really wants to do that because of the space, but you're able to really get a ton of density in those patch panels. And then I think even more important.

00:34:35 BOB

Is when you go to pull those cables through the pathways and with AI.

00:34:41 BOB

There's a lot of congestion as we talked about it 8 to 10 times more fiber pulling through these pathways, things get bound up and tied up.

00:34:49 BOB

Most of the big guys want pre terminated cable assemblies because they don't want their contractors on site terminating splicing because there's just a shortage of those people with the very overheated market we have today. So shipping a pre terminated cable assembly means pulling.

00:35:09 BOB

A cable with a bunch of connectors on.

00:35:12 BOB

It's through your pathways, much like you're trying to pull connectors through. You know, to do electrical work inside your walls. There's all sorts of things you'll get caught in there and you so you're fishing it very similar. You put it, wrap it.

00:35:24 BOB

Up it it.

00:35:25

Hmm.

00:35:26 BOB

It it bundles up and makes it a smoother, rounder section that pulls through easier, but if you can take.

00:35:33 BOB

Every one of those connectors and shrink them by 2/3.

00:35:37 BOB

It makes it way easier to pull through, so I think that if anything is where these VSF connectors really help out in these highly congested AI networks.

00:35:49 TODD

OK. So we've talked about a lot of different solutions. And so I'm curious how are these solutions combined together to future proof data centers facing rapid AI adoption?

00:36:01 BOB

Yeah. So the.

00:36:03 BOB

If you know you're going to go AI, and you if you're going to build a new data center, you should be betting you're going AI, because that's what almost everybody wants.

00:36:11 BOB

You first of all, you got to start with being able to have 4:15 in the building. You're going to want to lay out some sort of plumbing. If you're an MTDC Co location, that becomes a little bit more complicated because you have multiple tenants who some may want AI, some may not.

00:36:30 BOB

But if you're building a, you know a data Center for yourself, you should start putting that plumbing in now, because that will future proofing, whether it's going to be a liquid like water or one of the the the cool.

00:36:43 BOB

It shouldn't matter too much, but you can put all that in. You're going to want to plan to have your racks in a in a location that's going to be easy to scale out as well as scale up. Scale up. It means you're putting more and more into each rack so that those GPU's even even closer communication.

00:37:03 BOB

I would highly suggest using structured cabling as much as possible because as you add on.

00:37:10 BOB

Think about the cost of these these GPU servers. You may not be able to afford everything up front, but as you add on to it, if you put structured cabling in there, it makes it way easier to scale out, especially from switch to switch as you start to add maybe more layers of switching as your AI factor gets bigger and bigger.

00:37:31 BOB

So these are many things you gotta be thinking about for future proofing, and if you do have a raised floor, you should really make sure that your raised floor is going to be able to handle the.

00:37:42 BOB

Rates these AI servers are much heavier, the cabinets are being designed to be stronger, so today's typical raise floor may not be able to handle that sort of weight load on top of it.

00:37:56 TODD

OK.

00:37:57 TODD

Hey I I appreciate your time today and this is exactly why I wanted to have you on because you've talked introduced me to a lot of concepts I hadn't really thought about it around AI and the computing and the cooling and all the things that go into.

00:38:09 TODD

Feeding the data into those sexy machines that are doing all the blinking and light and computing. So thanks for that Bob. But I do want to closeout our conversation by refocusing on the why of what we do. So what motivates you day in and day out? What keeps you excited and passionate about what you do?

00:38:26 BOB

I would say it's always been about new products, new developments coming up with things that solve the customers problem.

00:38:35 BOB

And I mean, AI has been fantastic for me. It's it's been great because there are a lot more problems than it ever were before and it's it's really cool technology and the the what, the software systems can do to make people more efficient and like we're we're talking about.

00:38:55 BOB

Solving genome problems and and health risk issues that AI can hopefully really help with the just the humanity. It's it's very easy being motivated it it's been fantastic. I'm really loving being involved in the eye and so couldn't be happier right now.

00:39:13 TODD

Great. Well, Bob, thank you so much for bringing that energy and passion and thanks for being here and it's been.

00:39:17 TODD

Great having you on the show.

00:39:19 BOB

Thanks. It's been enjoyable to talk to you as well.

00:39:23 TODD

Well, that was my conversation with Bob Wagner, senior business development manager for data centers from Pandora. You can connect with Bob and Panduit to learn what they're working on by heading to the links in the show notes. I would love to hear what you're doing to support and build.

00:39:37 TODD

Out the AI.

00:39:37 TODD

Infrastructure. You can e-mail me at podcast@graybar.com or commenting on Spotify or.

00:39:43 BOB

YouTube.

00:39:44 TODD

If you enjoyed this episode, you can help us grow the show by subscribing and leaving a five star rating in your favorite podcast player. Thanks for listening to this episode of keeping the lights on.

00:39:53 TODD

We'll see you next time.

00:40:00 TODD

So do you have a favorite hobby media book? It could be anything, an activity that inspires you by that gives you energy.

00:40:09 BOB

Yeah. And I'll be honest. I wish I did it more often. You know, it's hopefully when I retire, I'll have more time. But my wife got us involved with the it's called the Hessen House. It's a local house where people go to when they don't have anywhere to live. And you know what's amazing is I think a lot of people have bad.

00:40:29 BOB

Thoughts about why people are in those situations? Most people just have no network and background that can. Someone can help them out and they just they just they're so grateful and they have like almost nothing. So helping those people.

00:40:44 BOB

It just it makes you feel good, but you also realize just how bloody lucky we all are. If we're not homeless. So I would love to do more of that. But, but yeah, that that's what really inspires me to be a better person.