Eric Dynowski CTO at ServerCentral Turing Group on the global ecosystem of cloud, managed services and application development.
Welcome to the tech deep dive podcast where we let our inner nerd come out and have fun getting into the weeds on all things tech. At Clark Sys, we believe tech should make your life better, searching Google is a waste of time, and the right vendor is often one you haven't heard of before. Hi. I'm Max Clark, and I'm talking with Eric Danowski, who is the CTO at Server Central Turing Group. Eric, thanks for joining.
Eric:No problem. Glad to be here.
Max:So, Eric, obviously, Server Central Turing Group. You came from Turing Group and are now an CTO for the combined entity. Give me a little background. Alright. Let's start with a little like, what was Server Central, what was Turing Group, and what do you guys now as as one one company?
Eric:Sure. Sure. I'll start with Turing Group since that's kind of my major background and and partly why I'm here partly with Service Central. So back in 2013, I was working in the financial services industry, working for hedge fund, managing kind of the whole team that ran all their global infrastructure. Started early on.
Eric:We built everything out for them, and I was starting to get a little little itchy, a little bit bored, and I decided to really take some risks and decided to start Turning Group. So in 2013, my business partner and I started the company with a focus on providing technology solutions based on, at the time, specifically AWS, but very solutions oriented. We believe that there was still a lot of companies that were moving to the cloud or starting to move to the cloud, but really doing it in the wrong way. They weren't embracing infrastructure as code. They weren't embracing automation.
Eric:They weren't embracing APIs. It was still like, let's boot a server. Let's install our software and, you know, run it. It was still very data center centric. And so we positioned ourselves as a company of experts that can help you leverage platforms like AWS as as they were intended to.
Eric:And so we, you know, challenged this idea that I think a lot of folks had that you move something into Amazon, it's just it'll automatically be redundant. It'll automatically be backed up, and it'll just auto scale because that's the way things work in Amazon. Right? And and the reality is none of those things are true. And you have to you have to design for those things and incorporate them into your strategy for using those services.
Eric:And so that's what we did. You know, we built that company up. Along the way, I had known you know, I've known Jordan actually since he started Server Central in in the late nineties, 1999, I think, 2000. And in the various roles I had been in previously, I purchased colocation space from Server Central. And so we've been in contact and chatting.
Eric:And when I started the company, you know, Jordan was kinda looped in on and I shouldn't probably introduce Jordan so everyone knows. Jordan is one of the cofounders of Server Central. You know, we've been in contact and talking about what we're all doing and and whatnot. And along the way, I had customers approaching me saying, you know, we love how you're helping us in in AWS. This is great.
Eric:But we have other needs that span beyond just AWS. We need some help either with network backhaul or colocation or bare metal solutions or things of that nature. And as Turing Group, we didn't have answers for those. And so my answer was, hey, Jordan. I got someone I think you can work with.
Eric:And, you know, that's probably a good segue into Service Central's history. So Jordan started Service Central in 2000, you know, doing primarily, hosting. And then as his customer base grew and his their needs grew, he, you know, evolved the company with Daniel Brask into a data center and colocation company initially. But in the same way that our customers were looking to, for lack of a better term, outsource things to AWS, their customers were looking to, you know, outsource more in house functions in terms of managing products and services. And so Service Central added on managed service capabilities.
Eric:So things like managed backup, managed storage, managed firewalls, managed private clusters, managed public cloud clusters, all based on VMware at the time, and things of that nature. And so, you know, I was kinda funneling some customers his way, and they were running into the opposite problem. They had customers that had signed agreements and and had maybe, you know, 50 cabinets and 200 kilowatts of power, bunch of bandwidth, and all kinds of great stuff, but they were starting to embark on their cloud journeys or cloud initiatives. And, you know, see their CIOs and CTOs were saying, hey. Well, how does Amazon fit in?
Eric:Or we have an initiative. And Service Central didn't have an answer, didn't have a way to continue to work with some of those customers in a way that was, you know, meaningful. So over lunch one day, we were chatting, and we're like, I think we have the same problem on opposite ends. And then one level, we might be viewed as competitors. But if we kind of bring this together, we can offer a much more interesting solution to our customers where we're not working against each other, but we're working with each other to build solutions.
Eric:And so in late 2018, we decided to to bring all that together. So we became Service Central Turning Group or SCTG for short so that we could work with some of those larger, more sophisticated customers that had use cases that necessitated both solutions on top of AWS, but also solutions within the data center and on top of more sort of classic platforms. But, also, Service Central liked that. We had sort of this infrastructure as code automation first approach to everything that we did. We believed in, you know, everything being completely repeatable.
Eric:You never did anything manually. Servers and instances should be ephemeral. You shouldn't treat them like your kids and nurture them. You should bring them up for a reason, use it, and then dispose of it and be able to recreate it on the fly. So we started incorporating a lot of that thinking also into the sort of broader approach to how we did everything as a company.
Eric:So, yeah. A lot lot there.
Max:A lot lot lot there. Actually, I'm I'm laughing about your your comment about, yeah, it's in the cloud. It's obviously redundant, and people won't have an outage. And I'm just thinking flashing back to, like, every US east outage in the past, like, 4 years. So this taken down massive swaths of the Internet.
Eric:That's right.
Max:I mean, redundancy in cloud's complicated too. Right? Because you're not just talking about, you know, having multiple instances running. Now you start talking about AZ redundancy, region redundancy, data replication. How does the application route, what goes where, where does it live.
Max:I mean, there that that's a whole can of worms as well that you start opening up when you start looking at these sorts of things.
Eric:Yeah. For sure. You know, what the public cloud provides is a set of building blocks and solutions that are distributed across multiple zones and regions. But you really have to factor that into your overall design and thinking to actually be able to take advantage of it. And you have to be pretty thoughtful about it.
Eric:It's really easy to think that, yes, I'm running 2 instances across 2 regions that I'm fully highly available and miss the fact that maybe you're using, one service somewhere under the hood that is a common single point of failure. In some ways, it actually creates even more complexity because it gives you that false sense of security. Right? Like, no. Yeah.
Eric:No. We store all of our data in s 3, and and we're connecting to s 3 just fine. And the s 3 has, you know, 17 nines of availability or whatever it is. And then US East goes down, and you can't even look at Amazon status page because they forgot about that single point of failure. You know?
Eric:So it takes some strategy and and, really complicated thinking into designing your applications correctly. And the other piece of that too is that a lot of enterprises aren't building their own applications. They're buying them. And they have limited options in terms of, you know, redesigning those apps to take full advantage of, you know, what the public cloud platforms can do, if you're in a position to control the architecture.
Max:One of my favorite examples I mean, you know, redundancy in cloud is it requires a lot of diligence. It requires a lot of, you know, energy and and continual diligence towards this. And Netflix was very famous for releasing their platform for this chaos monkey years ago
Eric:Right.
Max:Which literally randomly turns things off inside of their AWS environment. And when you wanna talk about, like, really taking this to an extreme of, you know, are we should be resilient and redundant? You know, we're just gonna randomly log in and select something running and just go boom and disable it and and Right. Does our system recover?
Eric:Right.
Max:I would imagine you're probably not advocating for that for most of your customers at this point. But, I mean, that is a pretty that's a that's a pretty amazing goal to have of saying at any moment, anything can be disabled randomly and the and the application is still gonna work.
Eric:Yeah. You know, I won't say that we don't advocate for it. I would say that we're thoughtful about when we advocate for it in the sense of where we at in the life cycle of the solution that we're building. And it doesn't really matter, actually, whether it's in AWS or something that we're building in our data centers or on top of our infrastructure. You know, if it's an early phase of testing and we really wanna test a lot of assumptions, we'll bring in some folks that weren't involved in the design that won't make the same assumptions and play chaos monkey.
Max:And
Eric:then it also depends on, you know, what our clients are asking for. And in some cases, we have had clients want us to test production systems like that, and we'll we'll do our best, in those situations. But the other the other interesting thing about it is creation, I think, of public cloud and the adoption of infrastructure as code and automation at scale has introduced a new point of failure, which didn't exist to the same scale historically, which is the engineer. And if you think back to the US east outage of s 3 that we were just talking about, that wasn't a technical failure. There was no design flaw in the architecture of s 3 such that, you know, it it caused that outage.
Eric:The outage was caused by, engineer making a mistake on one of the commands that they were running for performing maintenance. And what automation and infrastructure as code and, large scale control of a massive fleet of infrastructure does is it puts a lot of power to do something in the hands of 1 person. And I can't overemphasize, like, the need now for good operational practices, process, and control everywhere from change control to peer review because, you know, you can accidentally take out entire environments with a single command or a single typo in a configuration file. Whereas before, you know, maybe you had to log in to a fleet of servers or something like that. But now, you know, we have one customer we manage some IoT devices for, and there's thousands and thousands of them out in the field.
Eric:And one mistyped command, you know, we can brick them all. Jeez. Yeah. So so I think that the, you know, the advent of this approach to infrastructure has has created new new new challenges and new problems.
Max:So I'm gonna use a bingo word, which is, cloud transformation or digital transformation or customer journey. Right? You know, they call it kind of you know, they they get lumped in together at some point. And Mhmm. And at one point, Amazon and public cloud was new.
Max:And now we've got, you know, a lot of competition at sizes between Amazon, Google, and Azure. And, you know, a company, you know, is looking at these things of, okay, I'm maintaining data centers. Is this efficient? Or I've you know, I'm in the cloud. Is this sufficient?
Max:Or I've decided to move to the cloud. And and and it's it's still surprising how wide the scale of where people actually still are Yeah. This this transformation are. But, you know, let let's say I came to you, and I was still running, let's say I still had some on premise equipment, and maybe I had some IaaS with VMware Cloud somewhere, and and and we walked into your well, we wouldn't walk into your office anymore, but we we talked to you and said, hey, we wanna we wanna move this into public cloud.
Eric:Right. Right.
Max:You know, walk me through what that experience is like, you know, with with STG and how you take an organization through that process. Because Sure. You know, you you talked about it earlier. It's not just like, okay. Let's just replicate everything and and just turn it on over here.
Max:I mean, that's not a good end state. So
Eric:Right. Right. It's interesting that you asked that because, you know, we've evolved our methodology and our our process, you know, over the years. When we started Turning Group, I was a technologist. I I really enjoyed building things, and we would work with our customers and almost immediately jumped to solutioning.
Eric:Okay. Okay. This is what you're trying to do. Perfect. We need a we need an API gateway over here.
Eric:And, you know, what? We'll use EBS for that. We can use SQS over here. And, like, within 10 minutes of our of starting our engagement with our customer, we're, like, whiteboarding. And that was a lot of fun.
Eric:It was, you know, it was great. And we built some pretty cool stuff. But what we learned pretty quickly is as we moved into more complicated organizations that were, you know, decentralized, we walked into organizations maybe that had attempted migrations and failed, that had 22 Amazon accounts that they just discovered accidentally the other day they didn't know they had, things like that. We realized that we couldn't jump to solutioning right away. We had to really engage with the customer and ask them what problem they were trying to solve.
Eric:And you'd be surprised many times there was not an answer at, you know, at the tip of their tongue. And it's like, well, we're trying to do this, and we'd ask one more question. They're like, no. No. No.
Eric:No. We're well, our our CTO said we had to do it or or something like that. Right? And so what we learned over over the over the years was that we really had to take some time upfront to understand where our customers were at with their business, what the drivers were, how they arrived with the technology stack they have today, and, really, what problem they were ultimately trying to solve. Because the answer in some cases is, well, no.
Eric:We're trying to save cost. Okay. Fine. That that's that might be possible, but that's gonna have an influence on the design. No.
Eric:We're actually trying to solve a capacity problem. You know, we we need we need 500 servers, but we only need them for a month, and we don't wanna buy them. Or, we're trying to avoid having to build all this stuff in house. We really wanna just focus on our core capabilities and not worry about managing, you know, Redis servers or something like that. So we generally start most of our conversations now with the why, what are you trying to achieve.
Eric:And once we get to a a core understanding of that, that usually shifts onto an assessment of some sort. Meaning, you know, let's let's really get a good sense of of what you have. And I know we're talking very sort of public cloud focused at the moment. But in that sense, like, a lot of times, you know, customers would would try to do a one to one mapping. Right?
Eric:Well, I've got 27 servers on prem. Well, that's 27 instances in in, you know, in in AWS or Azure. That's not the case. Right? Or it's that well, no.
Eric:I have 27 servers, and each one has 8 cores and 16 gig of memory. Where if you do any sort of, you know, rudimentary analysis, you'll probably find they're highly underutilized. And so doing a one to one mapping, that way you're gonna spend way more money than you should. And then also if they're, you know, trying to solve for redundancy or scale problems, again, we're we're running into that challenge of them having assumptions about what the public cloud offers. So I think it's it's a very consultative approach initially.
Eric:And and we kind of help our customers in in the sense of just walking through them walking them through the entire process, you know, setting expectations about what kind of cost they could expect, setting expectations of what they can expect from a Doctor and data recovery perspective, you know, setting expectations in terms of governance. We've got so many customers that just wanna, you know, let people loose in the council, and then they're shocked when they get a bill for $200. You know?
Max:I I've I mean, everybody, I think, has that horror story at this point. Mean, it just comes down to how significant it was. I mean, I've I have one customer who, you know, they didn't realize it, and they spun something up to test it and made a configuration change. And it was $65,000 after 2 and a half weeks. And the capacity of I mean, it's awesome that you can spin up that kind of resource instantaneously and have that capacity, but then you get into whole governance.
Max:I mean, you mentioned that beforehand. Governance becomes very important as well. Like, you know, who is doing what, why, when, how, where, you know, and and what's the fiscal, you know, impact for it?
Eric:Yeah. Yeah. I I mean, there's it it it goes on and on. Like, we we had a financial services firm approach us saying they wanted to move, their all the they they had dockerized all of their applications, and they wanted to move everything to either Azure or AWS, but keep their database on prem on prem. And we did some analysis of the traffic, going in and out of their database, which they hadn't really thought about.
Eric:They just kind of were like, well, we've got, you know, 2 10 gig connections and low latency. It should be fine until you factor in the fact that Amazon's gonna charge you 2 and a half cents a gig.
Max:Yeah. From that direct connect. Yeah.
Eric:Yeah. Exactly. So once we did the math, it was like, well, your your, you know, compute's gonna cost you 2, $3 a month, and your bandwidth is gonna cost you 85. You sure this is still where you wanna go? You know, so, again, another reason to do kind of that thoughtful planning and analysis approach before, you know, just jumping in.
Eric:So in short, Max, we'd ask you, why are you trying to do this?
Max:You know, the bandwidth is an interesting point. It's usually overlooked in this conversation until you start getting the bills and you start digging into the I mean, Amazon bills are not easy to read. There's entire there's an entire industry just to help people read and understand their Amazon bills.
Eric:That's right.
Max:But, you know, egress costs are significant, and they build very fast. And this also has an implication now as we see I I feel like there was a rush to the cloud and this idea of, okay, the cloud will be cheaper, the cloud will be easier, as a cloud will be this or the you know, or the next thing.
Eric:Mhmm.
Max:And then now that we have organizations that started in cloud, you know, a decade ago, and are looking at it a decade later and realizing what they're actually spending, what are they spending it on, do they have a predictable app, you know, workload or cycle or things that are actually occurring. And now we're we're we're having conversations about, should it be in the cloud or should it come back off the cloud? Right. And that's complicated as well, but there is advantages to have, you know, okay, we'll use the market. You have hybrid environments where you have some in the cloud and and some.
Max:So how how do you guys navigate that? You know, what's is this something that you're driving to customers? Like, hey. We noticed that you guys are running, you know, a static workload effectively, and you should really think about doing this something different. Or are they driving the conversation?
Max:Or start with cost. Hey. We're spending a Yeah. Ridiculous amount of money. What do we do here?
Max:I mean, what's
Eric:it goes both ways. So I I think we probably have to first divide that into 2 2 different categories. Why someone is approaching us, maybe, you know, a prospect, and and what that conversation looks like versus existing customers that we have today that might be on AWS and looking to go on prem or or vice versa or looking to go hybrid. So for for existing customers, we try to engage and meet with them, you know, at kind of an executive and strategic level on a quarterly basis just to have some conversations with how things are going. We look at their utilization both in the cloud and on prem.
Eric:If we start seeing things that look kinda wonky or kind of interesting or if we start seeing bills that just where we know that they don't need to be spending that kind of money on something. You know, we have one social media customer that we've been working with that has been using s 3 to store petabytes of of information, but then they use s 3 as their origin source. And we are looking at these bills going, like, this is this is insane. Like, you're paying so much in egress fees that, like, there's gotta be a better way to do this. Right?
Eric:And and so we were able to have that conversation with them and say, well, first thing, at a minimum, if you put CloudFront in front of it, you can reduce your cost by 50% because transfer from CloudFront to s 3 is free, and you're only paying CloudFront, you know, fees. So so we were able to kinda come to them and and and sort of give them even just that little tidbit of information. But then we also said, well, you know, let's do that immediately because that's almost not even a technical there's almost no technical impact, and you can start seeing smaller bills. But, you know, we run our own on prem terabyte scale object storage solution, and our fees are nowhere near Amazon's, and we can give you the same price point. So unless you're doing something that's Amazon, you know, specific with s 3, like triggering Lambda functions or something like that, you know, we have a solution that's on prem that that that will perform to the same level, if not better, if you're in our region.
Eric:And and we can save you significantly a significant amount of money from what you're spending on AWS. On the flip side, you know, if we see customers are constantly pinging us saying, please add another terabyte of storage to to our managed, you know, cluster, or I need a new server. I need a new server. I need a new server. Like, if that's happening all the time, and then they're having conversations with us to decom stuff, you know, before the contract's up and we're just seeing, like, god, they don't they don't have that predictable workload.
Eric:We should probably dig in and understand what's driving that variability in their infrastructure needs and consumption. And in those cases, we might recommend saying, you know what? You're you're you're better off, you know, working in an environment where you can throw instances away on a daily basis, you know, and and we'll have those recommendations. Or another example might be that's not really related to performance was the company we were working with that has all the IoT products out there. In in one case, we could have gone on prem and developed an IoT solution from scratch, or we could have put them in AWS or Azure and use the IoT service offerings that both of them have.
Eric:In one case, the time to market would probably be 18 months. In the other case, time to market would be 3 months. That was a big thing for them. And so being able to jump on a platform that exists, that we know to work, and just get going right away was well worth any, you know, increasing costs that they might see because we can we can do that.
Max:Well, and that's a really good important point. Right? Because what you're talking about really is actually understanding the trade offs and spending money for velocity.
Eric:Mhmm. That's right.
Max:And, you know, businesses look at this from the standpoint of how long does it take us to get into market and what does it actually cost us? Are you are you allocating, resources in equipment, in hosting costs, in people, in whatever it is. Right? But, ultimately, it's it's you're investing in velocity. Right.
Max:And I think that's a really good example of where the cloud's great. Right? Because you can you can get into managed services, and you can experiment, and you don't have to commit to something, and you can Mhmm. Increase your velocity. But it requires you to take the next step, which is at some point, come back and look at it and reevaluate your decisions and say, is this the right thing for us still?
Eric:Right.
Max:You know? Right. And I see that a lot with, I mean, I don't know how many services Amazon has offhand. It's too many to count. Well, I mean, somebody counts it.
Max:But the
Eric:support It's well over 200.
Max:Yeah. It's it's ridiculous. And every week, there's more. Right?
Eric:But I know. It's terrible. But the
Max:problem that comes out of it now is you you get into situations like Elastic Cloud Elasticache, UCIN, or Elasticsearch. You know, there's there's a half a dozen dozen different ways of running Elasticsearch, not counting the products that are API compatible with Elasticsearch
Eric:that can
Max:run directly against s 3. And, like, each one of these has really significant price differentials for you and what these things cost. I mean, I won't pick on Elasticsearch, but, I mean, it's it's true for everything now in Amazon. You run it this way, it costs this much. You run it this way, it costs that much.
Max:And it all gives you the same thing, but at different scales. So that makes it interesting. Right? Because you have to constantly be looking at it and talking with customers and saying, what are you doing? Why are you doing it?
Max:Let's do this differently. You're in the cloud. Cloud has changed. Great. You know, something new has come out.
Max:Now it's time for your change.
Eric:Yeah. I I mean, I'll tell you the one thing that we're finding that's the one common denominator is companies are less and less interested in managing I I don't wanna say infrastructure because I feel like that's one layer below what I wanna say. But they're less interested in managing core infrastructure related application services. Meaning, like, they wanna use Elasticsearch. They don't wanna install it.
Eric:They don't wanna set it up. They don't wanna figure out how to shard it. They don't wanna figure out how to tune it. They just wanna use it. They wanna install Redis.
Eric:They wanna install Aerospike. They wanna install Cassandra. They wanna use all these different sort of core components that, you know, make up an application. Maybe it's, you know, ActiveMQ or or whatever it might be. And so where we kinda fit in in is that we understand where those things fit in in an application stack, what it takes to run them and operate them.
Eric:And where we're different than, say, AWS or Azure is that when your use cases start to, move into the perimeter to the edges of the normal use cases and those providers are no longer good for you, then we can give you interesting solutions. So, you know, it's you're only gonna take Redis so so far in AWS before either you hit performance limits or you hit budget limits. And and everything that a bit AWS does is geared towards that. Like, what is the most common use scenario from and and what are the most common performance characteristics? And once you start to get outside of those boundaries, then those providers no longer are a good fit.
Max:And and these numbers, I mean, the differential here is it's almost shocking when you see them the first time because it it's so high. I mean, Kinesis, you know, running Kinesis versus running different versions of managed Kafka versus running your own managing your own Kafka instances on top of EC 2. Right. We did a project for a customer, and I think the differential from Kinesis to them running their own instances was, like, 5 x. It was 5 x more expensive for them to run Kinesis and run their own infrastructure.
Eric:Right.
Max:And Right. You know, if that bill was $10,000 a month, you know, maybe it's not worth it for them to develop the you know, to devote engineering time, but, you know, they were spending a quarter $1,000,000 a month on Kinesis. And all of a sudden, that's that's a real significant number. I mean, you're talking about a substantial amount of of of salaries within the organization just in that one decision about, you know, this or or that. And
Eric:Right. You have to maintain the engineers to to run and operate that stuff and the expertise. And and, again, if if you're in that one particular you know, if you fall within that norm range, then, you know, public cloud is probably just fine. But if you start stepping outside of that, it doesn't make sense.
Max:Hi. I'm Max Clark, and you're listening to the Tech Deep Dive podcast. At ClarkSys, we believe tech should make your life better, searching Google is a waste of time, and the right vendor is often one you haven't heard of before. With thousands of negotiated contracts, ClarkSys has helped hundreds of businesses source and implement the right tech at the right price. You're looking for a new vendor and wanna have peace of mind knowing you've made the right decision?
Max:Visit us at clarksys.com to schedule an intro call. So now we're in a world of, you know, hybrid cloud. Right? Really, this idea, you've got some sort of physical infrastructure somewhere, and what does that actually look like?
Eric:Sure.
Max:And then, of course, multi cloud. People people look at multi cloud, and I have different opinions on multi cloud, but I'm kind of curious to hear what your your opinion on this is. But when you look at hybrid cloud and physical infrastructure, I mean, going back to your point of, you know, companies, they don't wanna manage it, and they're used to a consumption model. And it it becomes like, hey, I just wanna run containers. I wanna run Kubernetes somewhere.
Eric:Right.
Max:How is Server Central addressing that? And what has become your you know, at some point, you know, there's there's customization. But at some you know, but at at the same token, right, there is a certain, let's say, generic approach where most people fit into 80% of the same solution. Mhmm. And and and how do you approach that?
Eric:So first of all, I I wish more companies were further along the, containerization route. It does help move the conversation forward about multi cloud. I used to think that multi cloud was kind of a distraction. I used to think that our hybrid cloud solutions were were a distraction that if you wanted to really realize the power of AWS, you have to use AWS to its fullest extent. You have to leverage CloudFormation.
Eric:You have to use their their load balancer. You have to use their APIs. You've gotta you just gotta buy into the whole thing, and that's when you really can take advantage of what they have to offer. And if you say fold it in Azure into that mix, what you've done is reduce both providers to the least common denominator. And you're no longer getting the true value add that each of those providers can add and the special things that they have.
Eric:But as we are as as we sort of engage with more enterprises, the the notion of keeping or staying on a single cloud is is not tenable. And that's for a number of reasons. One is, the services that either that all the different providers provide aren't identical. They're not easily replicated across both sides. Right?
Eric:And larger enterprises are concerned about outages. You know, we've seen massive outages over at Google, and GCP problems when, you know, their routers went down, and we've seen significant companies go offline for it. You know, one of our customers, I think, broke their streak of, like, having, I don't remember, 5 or 6, years without a single outage. And because of Google, they were down. You know?
Eric:So there that that's a legitimate concern. I mean, actually, US I'm sorry. EU, West, and AWS today had issues. And so, you know, people wanna distribute their their risk. Also, I think from the hybrid piece, you know, we have customers approaching those that have significant stable workloads.
Eric:We're working with a social media company that has a need to probably I'd say probably close to 500 servers live all the time just to serve the base number of requests coming in at any given time. But they might have a celebrity on their platform, post something. That celebrity will have, we'll call it, 3,000,000 followers. And when when when she posts something, it generates a massive amount of traffic to update the feeds of all those millions of followers. And under normal circumstances, you might go a month and you don't need that and you don't need the compute you don't need the infrastructure to support that until that celebrity posts something and then all of a sudden things go crazy and they only go crazy for you know, 6 hours.
Eric:So scenarios like that where, you know, I can't back up an extra 500 servers in less than, you know, a minute, get them racked and ready to go and, you know, hand it off to you. Nobody wants to invest that kind of capital and sit on it when they don't need it. So things like, you know, hybrid solutions where, you know, I can spin up 500 instances in AWS in a matter of minutes and have your workload augmented and have that capacity available for you. To your point about containerization, none of that stuff is achievable really without containerization. And you have to really also be thinking about your application and designing it in, I think what is now being termed a cloud native way, meaning that it's it's very, very independent of underlying infrastructure or proprietary APIs.
Eric:And so kind of what's going on in the in the world of containerization right now is is the only way to achieve that.
Max:And we're seeing big plays. I mean, so, you know, we have Kubernetes from Google. Stuus has just bought Rancher Labs for several $100,000,000. I mean, there was a feeding frenzy for that one. There's there's expectations of I mean, we're gonna see Snowflake IPO here pretty soon.
Max:That's expected to be a big one. But Hashi Corp as well. A lot of people don't know Hashi Corp unless you're, you know, an infrastructure nerd. Right? And then all of a sudden, you're like, what's Hashi Corp?
Max:And they're gonna have their expectations pretty high as well. You know, it's interesting you talk about social media and this, like, kind of elastic capacity from the sense that this goes back from a year, so like the .com is the late nineties, and we had this idea that, like, every page had to be dynamically generated. And the reality was, well, no, every page didn't have to be dynamically generated. There were smart ways that you could do simple caching even if you were just caching for 60 seconds. It made massive differences to your infrastructure requirements.
Eric:Right.
Max:And you go, oh, well, it can be 5 minutes, and it can be, oh, well, if it's 5 minute cache, it's massive differences in your infrastructure. And that's really true also now with public cloud and and hybrid cloud and this and private resources of the cost differential is so massively different. You know, it's it's it's hard to really quantify that for people who will believe you. But, you know, like, oh, you're gonna spend 4 x. What you're gonna spend you know, AWS is gonna cost you 4 x.
Max:What it would cost you to have a a private environment? And no. That can't be true. No. No.
Max:It's really true. Amazon is gonna cost you 4 x and what a private environment is gonna cost you. But what does your private environment need to look like, and how do you Right. Maintain that? How do you size it?
Max:And now people don't wanna maintain data centers anymore, so how do you maintain that infrastructure, and who does those sort of things where you have have have turned into a a different conversation. And now, you know, managed services providers are are filling that void again and coming back in. It's it's this it's this very interesting dynamic where, you know, Amazon was killing all of the MSPs offering managed services related to data centers, but now all these data center companies are are really back in the front again because Mhmm. Companies realize, like, wait, we have to go back. Yeah.
Eric:I don't think we've seen too many folks moving back unless they're a really large player or they're doing a lot of traffic. You know, those are those are the things that are are really pushing customers back. What we're seeing is is there's still a lot of mid market just corporate world in general is still very much about public cloud and not ready to get out. Right? Companies that are are running just regular servers, installing their commercial off the shelf applications, running databases, running analytics workloads, We'll call them not Internet companies.
Eric:Right?
Max:Yeah.
Eric:You know, they get a lot of flexibility. What what they can do is really empower their employees in the sense of, well, now I don't have to call IT. I don't have to get budget approval for servers. I don't have to wait 4 weeks for them to arrive. I don't have wait for people to rack them and stack them.
Eric:I can just get started on my project. Mhmm. Right? And there's a lot of value in that, and corporations are still taking advantage of it. So I think that that initiative is still happening.
Eric:We have customers that are in the data center that are actively in the process of migrating to AWS and Azure. And the great part is is we're part of the conversation, and we're helping them. Where we're seeing that pushback, that move back from public cloud back into the data center is in the larger Internet companies. Anyone that's pushing a lot of media, you know, video, it gets expensive. Storage is expensive, and and the transit stuff.
Eric:Like, you know, I mean, that's part of the reason why we built our own managed object storage solution. Right? We we're hosting, you know, petabytes of of data on it because the companies that really need to use object storage to that degree, you know, the public cloud providers just don't scale in terms of cost.
Max:I I have a question here for you.
Eric:Yeah. Yeah. Mhmm.
Max:Late nineties, if you were an Internet company, there was a template from a from a VC firm. And the template said, in order for you to be a serious company, you had to run Oracle, you had to run WebLogic, you were running Netscape web server, and you were running all these things on Sun Servers. And that was the if you wanted to be a real Internet company, you had to have this template. And there were
Eric:That's right.
Max:And and it wasn't until there were some companies that started really breaking that mold. We saw, you know, eTwys. ETwys had, you know, mod Pearl with this custom caching layer and NetApp appliances. And it started releasing information around what their traffic profile looked like and what their relative infrastructure requirement was. And, oh, by the way, it's all open source.
Max:It doesn't cost us anything to run this thing. And then it became, oh, well, if they can run it, maybe we can run it now as well. And and and the mold kind of adjusted a little bit and became a little bit more adaptable, and other companies could follow suit. And I feel like that was the case as well with, you know, with with this conversation of cloud versus hybrid versus multi versus whatever. It was, well, we have this mandate.
Max:We have to go to cloud. There's no we have to be there because that's where we have to be. And it feels like that's loosened up a little bit where it's it's acceptable if you're not on AWS, and if you're on Google, or if you're on Azure. And it's acceptable if you're on 2 at the same time, and it's acceptable if you still have physical resources somewhere, you know, that have a strategic advantage for you.
Eric:Sure. I definitely think it is acceptable. There's probably areas of that, though, that are not. You know, I think if I was looking at a company and they're still running their own email platform, I'd go, why are you doing that?
Max:Well, unless unless they're an email business. Right? I mean
Eric:Unless they're an email business. That's true.
Max:That's true.
Eric:But if they're if they're, you know, a few hundred people company and, you know, call it a $100,000,000 of revenue, I'd be like, why are you running Exchange servers?
Max:As somebody who started out as an Exchange admin with an Exchange cert, I I I firmly support you on that. Nobody should be running their mail servers at all.
Eric:So I I think there's still some areas, within the the corporate IT stack that that I would look at and go, like, why are you doing that? There's way better solutions to it. And maybe it's not necessarily public cloud, but maybe a a SaaS provider of some sort. And and I think also, I would probably look at a stack and not necessarily think about it. You know, if I was a VC and I'm coming in to sort of do my technology due diligence, I wouldn't necessarily look at their solution in terms of, is it on Azure or is it in AWS or or on prem?
Eric:I would probably be more focused on the operations aspects of it. And what choices did you make in terms of deployment? How do you do your deployments? You know, what open source projects have you picked? What is at the core?
Eric:How is it architected? Is it containerized or not? How manageable is it? How scalable is it? And how much of a you know, versus a Frankenstein solution is probably where I would start digging.
Eric:How how supportable is it? How manageable is it? And how scalable is it? And not necessarily what platform is it on?
Max:The stats on Amazon or sorry. On Microsoft Azure, it's 50% 60% of their compute is running Linux and not Microsoft workload. Yeah. And that's Yeah. I mean, that's a that's a pretty amazing stat.
Max:You know? I mean, who would have thought?
Eric:I'm a I won't lie. I went from, you know, Microsoft was the board to I'm a huge fan of Satya and everything that he's doing. A 100%. He's very smart man. 100%.
Eric:Yeah. It's you know, they're they're talking about adopting Rust as the as their next programming language, you know, .netcorerunningonlinix. We just helped a customer build, we wrote the back end API for a a customer facing portal, and the customer facing portal is all written in dotnet containerized, which is great.
Max:5 years ago, you'd never imagine to have that conversation. Right?
Eric:No. Like like, 5 years ago, I would have been, like, oh, it's dot net. We're staying away.
Max:So so that's actually an interesting thing. We haven't talked about it all with this because, you know, a CTG is more than just you're not you're not giving somebody advice around managed services or SRE functions, or these kind of DevOps roles of, Hey, you know, Hey, we're gonna help you figure out how to run this in Amazon, or how to make sure Amazon's running right. You're actually writing code for people and helping people build platforms, or modify their existing systems or figure out, hey, you know, I've got this application that was running on premise, and now you wanna move it, and you wanna make it cloudy, and this is what we have to do with it.
Eric:That's right.
Max:Let's let's talk about that a little bit because, I mean, the the I don't wanna gloss over this. This is a really big deal.
Eric:Yeah. You know, that that that's a little bit of our the Turing Group roots that come through. You know, if you remember when we talked about when we started that company that we were effectively a group of of consultants. Right? We were technologists that had that were really smart about how to design and build distributed systems and and and do that within the public cloud.
Eric:So when I talked about how we first just jumped in and started talking about solutions without understanding the why, that trajectory led us to this place where the nature of our conversations with all of our customers changed. We weren't talking about gigabytes and, you know, how many terabytes of disk you need. We were talking about what problems are you trying to solve, right, and defining those problems and understanding them in a business context. And so our conversations now with many of our customers are are are exactly that. And then when they sort of learn that, hey.
Eric:We have an understanding around this stuff, they ask for help in building it and help in designing it and help in architecting it. And so in many cases, we get involved either as the development team, as part of a development team, or as advisors to a development team. But we we we also sort of act as advisors in in many other different ways. So you know, we had a a university approach us recently, and they had no idea how to pursue building, you know, a campus wide fiber network. But, you know, we have a massive, you know, fiber backbone and and and our experts in networking, Like, why why can't we leverage that and help them achieve their goals?
Eric:Right? So we're sort of unique and special in that sort of way in that we can act as this technology solutions company advisors, but we also have our own infrastructure and capabilities to support those solutions. So it's different from, like, a traditional consulting company in that sense, and it's also different from a managed services company in that sense. Sure. Can I give you a sales order for, you know, a petabyte of object storage?
Eric:I can. But I wanna make sure you're gonna succeed with it first.
Max:So, since since you went there, let's talk about this. Rust, Node. Js, Ruby on Rails, dot net, PHP, ColdFusion, let's put that out there as well. Python, you know, Scala, Java. I mean, that's that okay.
Max:Right? I mean, these are all still to different degree I mean, popular languages that are that are
Eric:important there.
Max:I mean, that's a lot of skill set to have. Is this stuff that you guys are dictating saying, hey. You know, we've got this project, and this should be in Rust, and this should be in Node. Js, and this should be in this. Or is it you're finding that the customers are saying, hey.
Max:You know, we heard that, you know, the hot thing right now is, you know, we wanna run GraphQL with with with you. You know? Can you help us?
Eric:Right. We will comment on language, if if a customer has not yet decided in in one particular language. You know, I think we're not dogmatic in terms of tooling, I would say. I would say we're dogmatic on architecture and philosophy and design choices. So in the sense that, you know, could you make this work in Node.
Eric:Js? You could. Could it be done in Go? It certainly could. Could it be done in c plus plus?
Eric:Yes. Are some of those better than others? Yes. They are. So we're I think we're very much in the opinion of the right tool for the job.
Eric:But I think we take other aspects under consideration. One is, are we developing it, or is your team developing it? And what does it mean for your business to have to I don't know. Let's take Pearl, for example, to support and run a Pearl application. Is Pearl a bad language?
Eric:You know, we can debate that all day long. Can you hire Pearl developers? That's not as debatable.
Max:Yeah. Right. And that's not even could you find people that know Perl. It's can you find people that know Perl that are gonna come work for you building Perl at this point? You know?
Max:And that's and I still use Perl almost every day. I mean, I just I you know, old habits die hard with certain things. And it's, you know, it's easy language to do certain things in. And and, you know, I find myself there all the time.
Eric:Yeah. Some languages are unavoidable in certain areas. So if we're talking about probably the land of, you know, infrastructure automation, you're either talking Python or Ruby, and that's that's it. Right? Like, you're not doing infrastructure automation in in c plus plus.
Eric:Right? And if you're doing it in in, you know, PHP, god help you. So I I think in in certain context, there's there's things like that. There's also technical considerations. Right?
Eric:Like, if you if you are thinking like, well, I really like some of the virtues of Elixir, but we wanna use Lambda. Well, there's no Elixir runtime. So, you don't have a choice. Right? You know, so I think in all those areas, including, you know, what kind of a pubsub or messaging bus should you use, We'll have opinions, and we'll help navigate those things.
Eric:And we're we're also happy to, engage in research and leverage our connections to to make sure that we're making the right recommendations.
Max:Environments went from physical servers to virtualization to public cloud. Right? Instances on public cloud. Containerization now, of course, I mean, if you're not containerized, you should get containerized as quickly as possible, disclaimer. And now there's a there's a pretty big push into, you know, serverless functions and function execution, and whether or not that in the form of an a, you know, Lambda or these different, you know, CDNs that are offering code run times on the edge.
Max:Yep. And what I've kinda wondered about with this for a while is not so much the, is this important and is this going to have a big change with how people actually architect their applications? But with containerization, you have a certain amount of portability that's very easy. And how much portability do you have with with a serverless function? And if you're making the decision to go down this path with AWS with Lambda, you know, even with all the efficiencies and neat things that you get out of it and things you don't have to think about.
Max:Right? Because you're not I mean, even even with Docker, now you're talking about you have to you have some sort of CICD pipeline that's building application, creating a container, pushing, you know, yada yada yada.
Eric:Yeah.
Max:I mean, you go serverless. I mean I mean, you really, it's like you have nothing. There's nothing you're you're thinking about. Is that in itself risky in terms of vendor buy lock in and and cloud lock in and portability?
Eric:Mhmm. There is risk, but I think it's different. And and I have two points to bring up on that. The first point is one of our customers, we actually built up a fairly large serverless application for them in AWS on top of, Node. Js as the core back end.
Eric:So it was API gateway. It was, Lambdas. It was some SNS, Dynamo, handful of other things. And that customer was courting a prospect in the retail industry. And that prospect said, we need in our contracts that you won't use any platforms on top of Amazon because we don't like Jeff Bezos, and we don't want any money at all going towards him.
Eric:So if you wanna work with us, you can't use AWS, basically, is what it came down to. Mhmm. And, you know, I felt really bad for our customer because their whole thing was built on top of Lambda, which is you know, feels pretty proprietary. And they asked us they they paid us to do some analysis of what would it take to port that to Azure Functions. And we're like, god.
Eric:These guys are nuts. No way. Like like, it's a it's gonna be a complete and total rewrite. And, like, for one customer, and, like, you know, we we we we did all our hand waving. And then eventually said, okay.
Eric:Fine. We'll we'll go do the analysis. And the result was surprising. While there was refactoring needed and there wasn't, you know, pure API compatibility between various services, the code that needed the least amount of refactoring was the Lambda code because 90% of the code within the function was straight up Node. Js, straight up JavaScript, nothing proprietary.
Eric:You could actually copy it and run it, you know, locally with the Node. Js command line. The code that go ahead.
Max:Yeah. I was gonna say, so you're talking about the code, the issue code being, like, Dynamo DB code versus
Eric:More of, like, okay. Well, you know what? The piece the pieces that talk to Dynamo need to get modified because Azure doesn't have a Dynamo. Right? Or the pieces that talk SNS.
Eric:In fact, actually, those were easy because, you know, you abstract most of that through abstraction layers. The more challenging area was authentication and IAM because those two things vary significantly, between the providers. So that was probably the most challenging piece. And then also we were using, in that case, the AWS IoT stack. We had to move to Azure IoT.
Eric:So there's a little bit of work there. But we walked the the moral of the story is we walked away thinking at the beginning, yeah, this is a total rewrite to I think we could do this in about 20% of the total time invested and reuse probably, you know, 80% of the original code. So it wasn't was it great? No. Was it the end of the world?
Eric:Also, no. It it is possible. But I think so that's that's one part of your your question about vendor lock in and risk and things like that. But I think the second piece that I like to think about and also why we exist is, you talked about we went from bare metal to virtualization. Now everyone's running on VMware to you know, the cloud and now, you know, containers and and now functions and all this other stuff.
Eric:And what the outcome of all of that has been has been a complete and total explosion of complexity because we have abstraction layer on top of abstraction layer on top of abstraction layer. And the developer that's writing Lambda code doesn't have any idea about how it's actually running and how it's talking to the network and how it's communicating. Right? There's so many levels of attraction and also so many tools. If we think about Kubernetes from, ingress utilities and processes There's Kubernetes is a is a orchestration of, like, you know, we'll call it 30 different processes that all have to work together just right.
Eric:Right? And from different companies and different sources. And what we're seeing is that our companies don't want to understand that complexity, and they wanna outsource it. They wanna outsource it to companies like us. Right?
Eric:They they don't wanna figure it out. They they just want it to work. They just wanna write their code. They wanna hit deploy and and just make it all happen. Right?
Eric:And so it's just and the level of expertise that you have to keep in house, you need to have people that know all these different things and know them really well and are nimble and are ready to evolve. Like, if you watch the, you know, number of products on the I think the Cloud Native Compute Foundation keeps a keeps a great chart of all the different technologies involved in developing cloud native applications. That's there's 200 things on there. Right?
Max:But so It's crazy. I okay. So there's some strong personalities in the Internet that are advocating for, monolithic applications again, like this return to the majestic monolith. Mhmm. And and, you know, at the same time, you see people posting their services oriented architecture, what what their actual application flow is, and we'll use, like, Lambda.
Max:Right? You know, we're we went functions, and and there's a flowchart, and there's, you know, 50, 60 things on this flowchart in order to do do something. Like, do and and yeah. Yeah. I mean, that that comes back to your point.
Max:I mean, it's interesting when you look at the complexity and the interactions and how do you maintain these applications and what are you actually doing with it. And I that comes back for me into the I mean, as you said, like, what are you trying to do and why are you trying to do it? You know? If you're just trying to send an email out, is this the right way for you to send an email out? You know, that might be not not the the smartest plan here.
Eric:You know, I think, Max, is that again, I I talk about this notion of being dogmatic about approaches in technology, and I think we always need to keep an open mind and understand what problem we're going after and then choose the right tools, technologies, and approach to get the job done. Secondly, I think complexity is like energy. It doesn't ever go away. It just transforms. Right?
Eric:You can never destroy energy. We just move it from one state and one way of being to a different state or way of being. And so, you know, maybe we had monolithic applications that were 2.8000000 lines. You know, good god. Good luck trying to find where the bug is in that code.
Eric:Right? To, you know what? We don't have any single app that's more than a 180 lines of code today. Good luck untangling the infrastructure and the transaction path between your functions to figure out where the problem happened. Now you gotta learn this highly complicated distributed transaction debugging tool.
Eric:So all we've done is move the complexity from here to there.
Max:Right.
Eric:So it's even in the Exchange example, we moved the complexity from our Exchange admin sitting in our office and running a cluster in our data center to Microsoft. Yeah. It's not simpler. Simpler for me today, but someone's absorbing the complexity.
Max:And what's really impressive about this for me, and it never really gets it never gets old. I'm always in awe of this is, you know, you went from the Internet being very nascent and coming on the Internet with modems and and this evolution and SDSL coming out and people having broadband. And and now, you know, phones with really good connectivity everywhere and this explosion of devices and data. And people used to call it, like, web scale. We're really talking about, like, planet wide civilization scale applications are available at your fingertips.
Max:Right. With with with now and and that the ability for velocity, this idea of velocity of very quickly being able to develop and roll things out and Right. And and produce things that can be consumed planet wide is is never I mean, it it never ceases to amaze me. I look at this with wonderment every time I think about it. But then I take a step back, and I say, okay.
Max:Great. So now you've built this thing. Now you have to make sure it's running. You have to keep it online to make sure you don't bankrupt yourself in the process. You know?
Max:Oops. We we had an s three bucket directly connected to the Internet, and somebody's downloaded all of our medical transcription files. And we just leaked all these patient records, like, because, whoops, we didn't configure. You know? And and Right.
Max:And that's the same thing where it's you talk about energy and complexity and being being shifted. So now it's become very easy, you know, an enterprise to use these tools, but it creates other complexity where they don't have expertise per se. And how do you keep track of which of the 100 Amazon services is the right one for your particular application? And, oh, by the way, did you secure it? Do you have a cost governance model around it?
Max:Are you tagging it properly? Can you figure out what's running? Who turned it on? Where it should be billed? Did should it be turned off?
Max:Like, you know, and I love all these tools that are now out there that do nothing other than just scan your AWS infrastructure and say, hey, by the way, you've got all these EBS volumes that aren't connected to anything that are charging $20,000 a month for.
Eric:Yeah. We just came full circle, Max.
Max:Yeah.
Eric:When we started the conversation, we talked about the idea that, there's a new kind of risk called the engineer. And the likelihood that they're gonna make a mistake is high because we have all this new complexity. So we can have a bucket that's exposed, or we can have you know, run a single command that affects a massive amount of infrastructure and makes the wrong change. Right? And then the second piece so there's all this complexity, and there's all this room for making mistakes.
Eric:And when we go back to the conversation about, well, when a VC is looking at your firm, you know, should they judge you based on what platform you're using or what technology you're using? No. It's about what are your operational practices? What do you have in place that reduces the chances that you will make mistakes, that reduces the chances that you're gonna fat finger something, that is a one off change, that it's not a documented change, that there was a change control review process put in place, that there's regular things that go on that ensure our infrastructure meets best practices. And it can't be a manual human audit.
Eric:It has to be continuous compliance. And that's the only way you you can sort of manage that risk.
Max:Well, Eric, I wanna thank you very much for joining. We could spend a lot more time together. Sure. Next time next time we're on, I think we're gonna have to debate different, orchestration platforms. I mean, maybe maybe I'll I'll have you pick Chef, and I'll be Ansible, or we'll do, like, Kubernetes versus Rancher, or we'll we'll we'll do something something Well,
Eric:that's not gonna end well.
Max:No. No. No. No. It's not it's not designed to end well.
Max:Right? No. But, Eric, thank you again. It's a pleasure. Yeah.
Eric:You're welcome. Anytime. Thank you, Max.
Max:Thanks for joining the Tech Deep Dive podcast. At Clarkxus, we believe tech should make your life better, searching Google is a waste of time, and the right vendor is often one you haven't heard of before. We can help you buy the right tech for your business. Visit us at clarksus.com to schedule an intro call.
