Trade-offs: Knowing When and Why You're Optimizing in Tech
I'm Max Clark and in 25 years of my career, I've had the pleasure of being able to work with some pretty amazing people and lots of them have changed the way I've thought about things and and looked at the world and how technology impacts it and what I want to talk about right now is optimization and knowing when and why you're optimizing. My best example of this that happened was, the beginning of 2,009. And we had a customer in our data center and they had ordered a new server and, and then a RAM upgrade. And so we were my team was doing this RAM upgrade for them, and it was a very large box. You know, quad socket, multi core, Dell server, and and we were adding I think it it was a 128 gig of RAM.
Speaker 1:I'll look for I have pictures of this somewhere. I'll find the pictures and try to include it with us. In 2009, adding a 128 gig of RAM to a to a machine was just an astronomical amount of RAM. Plus, you know, like a quad a quad socket, Intel machine was was pretty unusual. So this was a very big box.
Speaker 1:And I was, talking with a software architect and just, you know, was asking, you know, what what what are you guys doing with this thing? And and he told me that they were using it, to run paralyzed set and aux scripts. And, boy, this really made me curious. What do you mean you're to a paralyzed set aux scripts? This is kind of a okay.
Speaker 1:This is interesting. And, it it was an ad tech company, and they were taking and they had, you know, pods of ad servers. And these ad servers had to make a decision of of, what response to send back to the client. You know, a a piece of JavaScript or what have you know, of what to display, what what to just, you know, put in the browser. And this parallelized set not process job would take the logs off these servers and parse them and summarize them and insert the summarization into a postgres database.
Speaker 1:And it was doing this at an interval so that way that he could update accounts and know where their job performance was at and have they hit the ad targets they needed, with their customers? Have they not? Did they need to serve more ads? Which ads should they serve, etcetera? So this was this was the the pipeline that maintained all those back ends.
Speaker 1:Now in 2009, we've got lots of other things in the market. MapReduce had been out. So Hadoop had been for a long time, extensions to Hadoop, and with these systems like, like Hive had been around for a while. And there were a lot more elegant ways of doing this job than a paralyzed said NOC process. I mean, heck, even writing a you know, what today you would probably put into a Kafka process.
Speaker 1:Kafka didn't exist back then but implementing implementing a, a first in first out queue system, a queue, a message bus with, you know, some kind of workers to take and process that data was was readily available. And there was there was plenty of options in the market in order to actually accomplish this. Anyway, so I was fascinated and I was curious. And and I was I was I was so I was I was so confused as to why this would be the direction that they were taking. And what he told me was that by spending, I forget the dollar amount, but but he had a, you know, they just spent the money.
Speaker 1:They so they knew it. And spending a couple $100,000 on this machine and upgrading upgrading this, upgrading the RAM in order to run this process, which save him, 6 to 9 months worth of development. Now this wasn't saving him 6 or 9 months worth of developing other teams. It was just delaying the time at which they needed to do the development by 6 or 9 months. So he knew he could spend a couple $100,000 today and kick the can down the road and continue to focus on, building product to deliver to their customers to generate revenue for the business to grow.
Speaker 1:And it was a wonderful conversation. I'm I'm so grateful. You know, we had this talk. I mean, there's a lot more detail here. I mean, this isn't like a a 5 minute explanation of a, you know, hour plus long conversation in terms of how he was thinking about scaling the software teams, scaling product delivery, what they were doing for harbor selection, how they were picking platforms, why they were doing one thing versus another.
Speaker 1:I mean, we we really went around, you know, who really went around the horn with this one it was one of those moments though for me that really brought trade offs really to front of mind now and ever since then in terms of technology selection and implementation today the modern version of that comes in the form of cloud services. So this could be either an expression of an as a service type of solution. It could be a public cloud service. It could be a platform as a service. So there's, of course, infrastructure options where you can publish directly via a get check-in to actually deploy code without doing any things.
Speaker 1:You can get into a kubernetes style environment where you're creating, you know, containers. You could still be managing infrastructure on top of a cloud, and you can go down the stack and you can get closer and closer. And what triggered this, this this recording for me was a conversation I had recently with the CTO. And his frustration and and the quote was that he wanted to get closer to the metal. Now he wasn't talking about in terms of, like, cloud and he wanted to go from, you know, a cloud service down to, you know, running their own servers.
Speaker 1:He was talking about frustrations of dealing with layers of intermediaries and APIs and having issues with observability, performance, and cost. And so for them, as they're building and scaling their application, they were using an intermediary, which in this case, I I know the company very well And I know the makeup of what intermediaries that aggregators and resellers that company is using, and then who's the actual underlying, you know, providers for the service. And the discussion really came into whether or not it was time for them to, you know, go from here in terms of, like, you know, if if this is this is bare metal and this is the actual metal. Right? And up here and there's all these different layers in between.
Speaker 1:And and what each one of these layers represent is really additional cost and margin that that solution providers is is adding. What is the efficiency? What is the time to market? What is the ease of use? What is the there's there's some value that that in theory is being created.
Speaker 1:Maybe it's just you can't get access to the very bottom because you don't have the volume to commit to it. And so you need to have a reseller in play in order to get access to the system. Now that wasn't the case here. But the example is, you know, each tier, you know, as you go up through different resellers and intermediaries and aggregators or solution providers or whatever you wanna call this. In theory, in addition, you know, in exchange for charging margin, they're creating value for you.
Speaker 1:And then understanding and making a determination of where you wanna sit in the stack. Right? Do you make a trade and say, okay, I know that right now it's better for us to interact this tier. And, because it's gonna save us operational cost and overhead and complexity and things that we don't wanna deal with right now. And that's a great trade.
Speaker 1:I mean, if you're conscious and you're making that trade, make that trade. Make it take it every single day. Just be conscious of the trade that you're taking. And understand that there are gonna be times where it's gonna make sense for you to start jumping down the stack and moving farther and farther down. I see this a lot right now.
Speaker 1:If we talk about you see a lot in telephony. So anybody that is interacting at some point with a PSTN interface or if you're doing messaging, whether that's SMS or WhatsApp or Apple Business Messenger or these different, you know, these different platforms. But you have an option of interacting and and going through the energy of of going all the way down. In this case, going all the way down to the metal, we'll call it, or or finding it intermediary and working your way up and trying to find ease. And whether that's just, again, volume commitments or maybe you just you you know, you're not at a position where you can sign a contract to commit to something, that's gonna put you up here.
Speaker 1:And that's fine. But then when you're at a point that you can commit to something, you can jump down 1 or 2 spots. And and this is a a great place as you're evolving your operations and as your business evolves to really understand and and and kind of keep that in mind of of when it makes sense to jump down. And by the way, jumping down usually also creates other advantages and strategic differentiation for your for your business, for your product and service. And if all your competitors are running up here and you figure out how to go down 3 levels, you can take advantage of of that of that change.
Speaker 1:So in the in the compute and infrastructure world, right, we you know, I I talked briefly about platform as a service. And when I say platform as a service here, what I mean is is, again, you know, these these systems where you can just do a git push and it orchestrates the entire infrastructure for you. Right? So everything is a black box. You have no idea what's going on.
Speaker 1:And there are wonderful, easy environments to use and develop with and be quick with and not have a big team around it. And then you go from there into, you know, what was, like, what we consider public cloud. Right? The hypervisor environment, the hyperscalers. I really don't like that term, but the public cloud environments, the AWS, GCP, Azures, or maybe, you know, a Linode or DigitalOcean or, you know, Oracle or IBM or whoever whoever is there for whatever particular reasons you're using them.
Speaker 1:And then you get into a position where you start looking at and talking about, is it do you wanna go serverless with those platforms? Did you go down a container environment with those platforms? The trade offs of serverless is now you're building code that becomes very specific to that platform. Serverless is not a technique to give you portability between cloud providers. Containers are wonderful, though.
Speaker 1:Containers, in theory, can run anywhere with with relative ease. So if you're if you're looking at this and you have, legacy infrastructure, legacy code, legacy applications that are still running on, compute instances and you're trying to figure out where to go from there, it might make sense for you to go to contain to serverless, I would tell you, it probably makes a lot of sense for you to go to containers as a next step anyways because that unlocks a lot of flexibility for you. And then once you get into containers, now you can start making other decisions. Do you stay in that public cloud environment or do you move your container and your containers out into something else? And I I find this misconception that this moving out of public cloud means moving back into the stone ages of what we did, you know, 10, 15, 20 years ago of of having to go and and build out your own data center or or or going into a colocation environment buying your own servers.
Speaker 1:And that's absolutely an option to you. You can, you know, we can we can help you get a colocation facility, and we can help you purchase servers from a bunch of different, you know, OEMs and ODMs and help you manage it. And it's not hard. It requires a different muscle and operational process for you to get into. You just you know, if you haven't had experience with it, of course, it's scary because you've never done it.
Speaker 1:But, you know, once you do it, it's not that bad. You know, you have to keep track of equipment aging and, you know, hardware refresh cycles. And, you know, you don't just buy a server once and it's gonna run for 20 years. You know, there is some level of a of a refresh cycle that you have to get into, and that trips a lot of people up because they don't really plan for it. But that's all the way over here.
Speaker 1:And there's lots of options. You know, if public clouds over here and and running your own data center, buying your own services here, there's still a ton of options in between those two points. You can go to you know, there's there's options for managed data center. There's options for managed servers. There's VPSs.
Speaker 1:There's bare metal providers. There's managed Kubernetes providers. And the trick now becomes, you know, at what again, you know, we're talking about this stack up and down. I'll go I'll stick vertically. Just where you wanna sit within these things.
Speaker 1:And the point is is with each level that you come down, in many cases, what you're doing is you're adding, operational overhead to you that that you don't have a lot of operational overhead with with AWS. I mean, you do. You know, it's not 0. You still have to have a team to manage it. Right?
Speaker 1:But you're not driving to a data center and changing hard drives out. So we'll be clear about that. So you're adding your trading operational overhead, but what you gain in terms of that is is cost and cost differential. And, I've talked about this a lot. I won't I won't beat this one to death.
Speaker 1:But, you know, the cost to compute is is really, it's really the distraction prop in cloud. You know, everybody gets focused on compute. And we can tell you all the numbers and difference between, you know, purchasing our eyes and AWS versus, you know, buying servers and co locating them. But the cost of compute is really the distraction for, you know, the real, you know, cost suck in cloud, which is, your bandwidth costs and your egress costs. I'm not gonna go down that right.
Speaker 1:I'm not gonna go down that road right now. Anyways, the point of all this, again, to circle back and start at the beginning, is just understand what trade offs you're making. And it's not bad to say, hey. You know, we wanna be here, and we know why, and this is the trade off we're making. That's actually a a great place to be.
Speaker 1:Just be conscious of what you've decided and know that any decision you make shouldn't be a permanent decision. In 2 years, you could be in a completely different situation and and need something else. That's okay. And just get ahead of it and use it to scale your your business and scale your operations and, you know, refine your service and get better and build modes and and have differentiation and, you know, create competitive advantages and everything else you need to do in order for your business to, you know, to succeed and win. So I'm Max Clark.
Speaker 1:A quick rant on trade offs and infrastructure, and I hope this helps you somehow.
