DevOps approach is gaining momentum because it tightly integrates development and testing with system operations.But how can the two groups work more closely together and automate the deployment process. How can dev teams make sure the changes map to the larger initiatives at the business level? How does the organization address the change management issues and focus on competitive differentiation and customer retention?
John Willis, Co-Author, "The Devops Handbook"
John Willis is the Author of “The Devops Handbook” which he co-authored with Gene Kim and Jez Humble. He currently serves as the Director of Ecosystem Development for Docker, which he joined after SocketPlane, the company he co-founded,... More View all posts
John Willis is the Author of “The Devops Handbook” which he co-authored with Gene Kim and Jez Humble. He currently serves as the Director of Ecosystem Development for Docker, which he joined after SocketPlane, the company he co-founded, was acquired by Docker. Previously, John was the Chief DevOps Evangelist at Dell. He has also held past executive roles at Opscode/Chef and Canonical/Ubuntu. Less View all posts
Leonard J. Hardy, Senior Vice President of Operations & Technology, Northern Trust Corporation
Leonard Hardy is a Senior Vice President at The Northern Trust Company, Chicago. Len is the Director of Architecture and Innovation and the Chief Architect at Northern Trust. He is responsible for development and implementation of IT solu... More View all posts
Leonard Hardy is a Senior Vice President at The Northern Trust Company, Chicago. Len is the Director of Architecture and Innovation and the Chief Architect at Northern Trust. He is responsible for development and implementation of IT solution and enterprise architecture strategies to unify and standardize deliverables across the Northern Trust’s global technology organization. Architecture and Innovation (A&I) also champions Research and Development efforts to identify innovation opportunities and their associated business impacts, partnering with other IT and business stakeholders to determine business value and impact/benefit to development partners. Migrating new and innovative technologies into the approved Development Patterns where appropriate is also a responsibility of A&I. He holds a BS degree in Computer Science from Northern Illinois University and has 33 years of experience in the industry. Less View all posts
Sanjog: Hello and welcome to CIO Talk Radio to learn more about the show please visit www.ciotalkradio.com. And as always would like to invite you to join the discussion on Twitter and look for this share show as #DevOps. So today’s topic is creating a DevOps vision and roadmap and our guests what today’s show are. Leonard Hardy who’s a Senior Vice President of Operations and Technology with Northern Trust Corporation. Hi Len, how are you?
Len: Good morning Sanjog. Thanks for having me.
Sanjog: Oh great to have you. And we also have John Willis who is the co-author of the DevOps Handbook Hey John, how are you doing?
John: Hey. Great. Good morning.
Sanjog: Good morning to you as well. So the topic today we picked up because of course DevOps is not brand new to us, it’s not any more hype or something that we should be just concerned about people are trying to implement but then we also see that it has become a little tougher than what people otherwise thought. Okay. You’ve got operations group and you got the development group and we have to get them together. So earlier it started as, something cool but now it’s becoming a huge change management initiative. There are pains, there are challenges, there are myths, there are mysteries. So what we wanted to do today, is to look at to DevOps objectively and see what would it actually take for us to create a vision for DevOps and then subsequently what would that playbook look like so that it really becomes something you can create and leverage.
So that said Len, if I were to ask you, are there multiple definitions or interpretation of this DevOps even today?
Len: So potentially, I’ll tell you what the definition of DevOps is at Northern Trust. To us, it’s all about removing or minimizing the things that the developers traditionally views that propose to their productivity at the time the market, right. So today, these things exist to do operational process functions tasks and again, to us DevOps is really about enabling continuous integration, continuous delivery, automating as much of the integration testing and code scanning. And also migration task as we possibly can. These are the things that have historically been manual and time consuming and processed by operational staff. I mean really the bottom line is that all about, bringing business value and bringing up to market faster.
John: I’ve been 35 years as what I call IT operations junkie. But about eight years ago I ran into Luke Kanies who’s a founder Puppet Labs, he was a one man show. I started working with him. Started then, then became an early ad on and chef. It was one person in chef but about seven years ago, six years ago, it was the first DevOps days. And I actually was the only American to actually attend the first DevOps days in Belgium. In the early days, we said this whole question about the definition of DevOps, in the early days we said, we went by Adam Jacobs, It’s a cultural professional movement full stop. Don’t argue with it. Deal with the culture, it’s a professional movement, learn from everything that’s going on in the ecosystem. A little bit later at Damon Edwards, my co-host at DevOps cafe podcast came up this camp culture automation and sharing. But long story short, over the last three years, I’ve been working with Gene Kim and Jess Amble and Patrick Barnes, DevOps Handbook and I really studied a lot of what Gene has been doing in IT revolution, Gene author of the Phoenix project what we consider high performance organizations. And I think today, literally we just were like code complete on DevOps handbook like this week. I would say that what DevOps really is this idea and I agree with what you just said but I think it’s about– honestly turning human capital, it’s a methodology for turning human capital into high performance organizational capital. And I know that’s a mouthful but it really when it comes down to, we can study what Toyota did, what Lean did, we can dive down into the tools. But at the end of the day we’re taking this thing that we have with human capital, which is this gray area on our market cap, if we’re a public company and organizational capital that we can see as high performance. So it really is, it is about that glue in between humans and what they can do and how do you turn that into this visible high performance organizational capital.
Sanjog: So Len, when you look at the organization that they you were trying to develop, I’m sure this also started a number of years ago in your organization. And when you mention about automation and optimizing resources and how to make it seamless, that’s actually goes back before the DevOps term was even coined. So what is the incremental tweak if you will that you’re doing or is this still a transformation for you?
Len: So again from Northern Trust perspective, I think that we are just really starting what I would consider the DevOps journey. We’re 125 year old company. We’ve had applications in production since the early to mid-60s. We are just beginning this transformation if you will and what brought that about was really, we understood from a technology perspective that we’re going to meet our corporate goals which I think are pretty common to other organizations, better client satisfaction, faster speed to market, better employee satisfaction, higher ROI NV those kind of things that we needed from a technology perspective to deliver better applications, more secure applications and we needed to deliver that business value faster. So we were looking at ways to transform and revolutionize the way that we’ve been developing software here for quite some time. So I would say at the very beginning of what we’re considering the DevOps evolution at this point.
Sanjog: So when you beginning, so that means you have a blank slate is that what you’re saying because of course you had some automation and some tweaking and some uniform integration if you will between DevOps that was already happening in the past?
Len: Yes, I guess what I’m saying is we’re trying to automate a lot of the things that were done manually. You guys have to understand we are a financial services industry and we’re very, very highly regulated. We have auditors and regulators in here all the time. There is a lot of process and procedure around what we have to do with moving quarter production and things like that. So what we’re trying to do is figure out how we can automate some of the things that have been manual in sort of the past. And still meet the requirements of the regulators and the auditors and the Federal Reserve and other bodies like that both here and in Europe and in Asia because we are a global company. At this point, we have offices in 20 international a patient in 19 states, so we have to do things in a way that are going to meet the requirements of each of those different jurisdictions. So I guess, just to sum up where I think we are, like you said, we have a lot of legacy. But we’re undergoing a large transformation right now. Some of the things that we’re building in redeveloping are in fact Greenfield. So we are in the process of trying to figure out how we’re going to integrate DevOps into Greenfield. Plus, how we’re going to start moving the needle from our legacy environment into that same DevOps environment.
Sanjog: When you look at, John, from your viewpoint because I’m sure you’ve looked at the companies that you may have directly worked with or as a trend and as you wrote your book. What are the typical trends, what is that inflection point if you will, where people say, the regular stuff is not working, let’s go the DevOps route and they do see that it brings some significant ROI for all this effort that they put in?
Len: Yeah, Len’s experience is very similar to almost every company. We did this DevOps Enterprise Summit and we have about 50 or 60 large companies. We had like. Sherman Williams, he went talk about– coming up talking about what they did at DevOps or Western Union but we also have banks, Barclays and Target, Nordstrom. But it’s not a one size fits all, I think it is a journey like Len said. I think Greenfield– I’m kind of reconfirming what Len said. You look at Greenfield opportunities, you look for a quick wins to prove out theory. To get deep again, and lot of what we’ve learned over the last couple of years about DevOps is, a lot of it comes from mean toward a pressure system and to go real deep Dr. Deming. But in the end, it’s experimental. Like you don’t just read a book and say, “Oh, I’m going to do that.” You read a book and you say, “Wow, I like what Target has done, I like what Nordstrom has done.” And then I say, “Let me try this out as an experiment.” And I actually treat it like a science experiment so I actually implement things with the test markers. I mean Deming said, plan, do, check, act. And Toyota lived and died by scientific method. The original hard business review, decoding the DNA of Toyota pressure system, Stephen Speirs said, Toyota was a community of scientist’s continual experiment. So I know I get along with it but basically your journey is that you try things. You probably don’t take a 40 year old legacy application as your first strike. You take a Greenfield, maybe parts of a Brownfield structure that look like, you can turn that into an experiment. Okay. We’re going to use these type of tools, we’re going to measure these type of things. And we’re going to put that in play and we’re going to find out does this work for us. So, the answer might be your culture is so bad that even this experiment would work for a hundred of a hundred people that gave presentations about it, didn’t work for our organization. Why was that? And then you apply kind of that scientific method to figure it out. And maybe you five why your way back to find out that we’ve got a broken culture.
The one thing I do want to also point out which is really important. The DevOps survey has been going on for about four years now. This last year was really significant. And we’re actually starting to codify and this is lot about what– I’m trying not to sell the book. But it’s a crate book and I’m just one of four three or four authors on it but what we found through the DevOps survey that has been run through Gene Kim in Puppet Labs, is that it boils down to two metrics. One is lead time, how agile are you? Can you measure your ability to deliver from white board to what we call the aha-tic chain. How good are you? We are seeing significant differences between what we call high performance organizations and low performing organizations. And how good are they at lead time, which is how often do you deploy, how agile are you to deploy, how many times can you deploy, do you have waterfall? And the second metric which is just my favorite all time, is how good are you at repair? So those two which we call mean time to repair or mean time to recovery. And what we’re finding is that out of all this DevOps book that we did a whole industry which is great because I remember when there were 30 people talking about this six years ago as the word DevOps, not in general. But now thousands and companies and everybody is going down to this thing but we’re finding our two important metrics is how good are you and can you quantify how good you are at delivering it lead time. And how quick can you repair things? And we are finding is high performance organizations are just extremely better at those who metrics then the ones that are self-defined through statistical surveys that we’ve been doing as low performing organizations.
Sanjog: Let’s take a quick break listeners, we will be right back. And this is interesting where we are looking at surveys and other form of quantifiable metrics to see whether an organization will be able to effectively leverage DevOps. And are the culture would really be conducive for it? So let’s see if it is truly a data crunching or gut or an art or a science completely. What’s that combination? What’s that sweet spot that an organization should be ad where they can effectively take DevOps head on and really tame it? Please stay tune listeners, we will be right back.
Sanjog: All right, so we’re back and John, based on your last comment I’d like to ask a question to Len, that in your organization, when you are working with numbers or you’re getting started and taking some greenfield opportunities to test out this DevOps would you really do numerical analysis of the state off where your organization stands or is it going to be a typical leadership and finding yourself through the haze, finding your way through the haze is the approach that you would have to take in order for you to see DevOps is, are you ready for the DevOps?
Len: So, my perspective is like most things. It’s a combination of both. Even in today’s environment, even in your yesterday’s or last month’s waterfall environment, we have all sorts of metrics that we take in terms of up time in terms of application failures and perhaps with some of the things that John mentioned mean time to repair, mean time to restoration, things like that. So we do have metrics that we can compare. We know how many times, we migrate to production for each application a year and we know how many times, we’re doing it more and more, more actively now that we’re started down sort of DevOps journey. So again, it’s just like everything else, it’s not one thing, it’s about the people, it’s about the process and it’s about the technology. And some of those things you can put metrics around and some of them you have to go by gut, I think.
John: I mean, again, what works, works, right? Like having mean potatoes, right. I look for the things that work in patterns. Somebody tells me that they’re improving– great. I would say the successful patterns are where you decide, back to what I was saying earlier, it’s not whether you’re measuring ongoing. It’s not measurement and I’m not saying that you did this but I’m saying measure it, measure and say, great. Without applying some type of hypothesis to things. So what we’re finding is and this comes a lot from like Eric Ries’s Lean Startup, Jess Amble one of the coauthors on handbook road, Lean Enterprise. Where you start thinking about things very much, if you look at everything, Eric Ries has said about Lean Startup is, it starts applying to the enterprise. In the startup world, the really, really good web properties are actually putting something in front of a customer, testing the result, pivoting or moving forward or triple or double down on it. And what we’re finding is these models work really good in the enterprise as well and again, I will talk about the DevOps Enterprise Summit where you have Nordstrom and Target. There was about 50 companies but the point is, so you’re going to do a Greenfield. So instead of just doing a Greenfield like, okay, it looks like company actually use chefs, so we will use chef, it looks like ad company actually use Jenkins, so we’ll use Jenkins, it looks like company actually use Splunks so we will use Splunks. So, let’s just flat those three things down and start doing DevOps. And then six months later you find that your culture and nobody really knew how to do it really, so chef didn’t work.
And I’ve seen it. I have seen large companies that kind of did the false version of DevOps where they brought in these big products, it was kind of a cargo cult, it looked at other competition, lot of other chef, use this and put it. And he didn’t act like scientists. And the ones that are successful say, you know what, why do we want to see outcome? We’ve got a Greenfield. We want to take this Greenfield that we want to get certain kinds of results. So let’s put the measurements on what kind of results we’re going to get from this. As the hypothesis is we’re going to do X, Y and Z and we’re going to see these three outcomes. And if we see those three outcomes then we build further. We don’t see those three outcomes, we pivot. And creating this pivot or process, Mike Rother’s Toyota Kata, which is excellent. One of the best books I read in 2014. Talks about this, basically improving Kata. And how Toyota would just everything was really in this scientific loop. And I know I’d actually sound like a geek but I’m telling you that the pattern of success that we’re seeing from Gene to Jess, some of the leadership now in the DevOps movement is based on this kind of model of– And again, going back to those two metrics just keep coming up all the time. Lead Time and MTTR.
Len: I’d like to make a couple of comments on that. So John, you talked about sort of the concept of, fail fast, fail up and pivot those kind of things I think it’s interesting. There’s a quote that I saw that I thought was really interesting and it was by somebody who’s not associated with DevOps, in fact it was from Thomas Edison who said, I haven’t really failed, I’ve just found ten thousand ways that won’t work, right. I think the concept of sort of the laboratory approach and being able to fail and the fact that people have to understand that that’s the way that it’s going to work, not everything is going to work the first time and you have to be nimble and you have to be agile and you have to be able to pivot and you have to be able to change your perspective on how you’re going to deliver those things. I think that’s very important.
And again, one more comment on your comment about the hypothesis, right? So what we’re having a hard time doing is convincing some of the people that I mentioned earlier so some of the regulators and some of the auditors and those kind of things that a DevOps migrate way more often than you did in the old way is actually less risky. And our hypothesis is that it’s less risky. And yesterday’s world if I had a big monolithic application and I had a big project on an application, I might migrate to production three or four times a year. What do you think the mean time to repair is, if there’s a problem with that when that developer may have worked then that specific enhanced it which was 1 out of 500 or 1 out of a 1000 four months ago. They have no idea where that code is anymore or how to go in and fix it. The mean time to repair is sky high. Now if I’m migrating that application once a week, once a month, once a day something in a very, very quick schedule, everything is clear and the developer, the architect and the testers mind and they have a very, very good idea of where that code is and how to fix it. So the hypothesis is that mean time to repair is going to drop significantly and there’s going to be less risk in the DevOps world than there is in the old waterfall environment.
John: Len, spot on. I mean I’m [unintelligible 00:22:21] about failure too, and you’re right. Embracing failure, understanding failure is just the– Mike Rother says, if you look up the word failure in the dictionary, it doesn’t say, get away, run, stay away from this page. It doesn’t really have a bad connotation, it’s just the output of an experiment. And to think about failure too, the other point is you’re spot on about treating things is failure is that culturally you have to create a model where people don’t get chastised for failure. That’s one of the biggest problems we have, is changing this culture. That’s why we talk about things like blameless postmortems. This people actually will teach you how to do blame as postmortems where you don’t say, should’ve or would’ve or could’ve– you can’t use those words. You don’t identify people, “No, Bill broke the–” You think of things a system. So, there’s a whole– we could spend an hour on the culture of embracing failure. But again I go back to that thing, I think you were spot on is this is why I love MTTR. In the DevOps survey in 2015, we found that the difference between a high performing organization and a low performing organizations, was a 168 times better at MTTR. Let me say that, 168– they were 168 times better at repairing. So when they did more frequent deploys, they statistically– we had a PhD statistician Nicole Fortran, worked on the psychometrics on how to do the surveys. So this is real stuff. And we found that the people who had these patterns of delivery were 168 times better at repairing their systems when they broke than the people who were considered low performing. And that comes from moving from waterfall to wrap it innovation, small batch. So, exactly spot on Len.
Sanjog: So, we spoke– both of you spoke about sandboxing. So like anything new of course you would want to do a little bit of that but then how much is the appetite for the enterprise which is saying, we anyway are trying to reduce their cost. And yes, you tried to promise– you actually promised to reduce the cost as a result of making it more homogenous in terms of DevOps and then you want money for it. But then you’re still sandboxing, till what time will you do that before you say, “Okay, this is my unique model that’s going to work for my organization.” Len?
Len: My perspective is and again, a company that’s been around 125 years is, I don’t think that we will have one model. I think that– I guess Gartner calls it Bimodal IT, I think that IBM might call it Multimodal IT but. I guess the bottom line with that kind of theory or strategy is traditional IT is necessarily going away, especially for the large enterprise. So, in Northern Trust, we have both core processing applications, we have client basing internet and mobile application– from our perspective that all workloads are created equal and some are more easily lending themselves to edge development in DevOps. So I think this concept of bimodal or multimodal, both traditional and IT and agile and DevOps are required by our organization and from our perspective, each are equally important and each are considered to be equally important. We don’t like to segment resources here to say, you’re mode one, you’re mode two or you’re service oriented or you’re in the laboratory or things like that. We don’t want to label or silo people. So given the chance, our perspective is people will self-select where they’re most comfortable and most productive.
I guess, the answer to that question is, is we don’t see a 100% full migration to a DevOps environment, at least not for Northern Trust. We see traditional IT having its place. We see agile, we see rapid development, we see DevOps having its place also.
John: I just want to add. There’s a great story that Nordstrom did at DevOps Enterprise Summit in 2015. And I hate Bimodal, multimodal – There is no one size fits all. Like Len said, there is no two modes. You know this is why I hate Gartner, I don’t waste too much time, I hate Gartner, there are two modes, okay, well, and it’s that simple. No, it’s not that simple. And here’s a great story, right. And the other thing, it’s not always about tools. DevOps is not just tools. I mean tools are part and point. The story from Nordstrom is amazing. So they were talking about, they’ve gone on a lot of Greenfield, a lot of do API development that stuff, full in on DevOps, they’ve been doing this couple years. One of things that came up all the time was they had a mainframe COBOL application. I’m old enough to know, I’ve been doing this 35 years. Like I worked on Postings years ago. Mainframe COBOL and every year, at some management level will come up like, “Should we get rid of this?” Well, you know it’s been around for 40. Nordstrom is a 115 year old company as well. And so they’ve come up and people say, we’re just getting rid of it, we get off mainframe, it’s killing us. And one of the things that Courtney Meszaros, one of the head– chief, I forgot her title. But she’s been doing a lot of transformation at Nordstrom. They applied something that they are doing in a lot of the web based business, the API stuff, it was something called value stream mapping.
And value stream mapping is incredible tool, it started with Toyota. It is used quite often in Mike Rother wrote a book Learning to See. It’s a valuable tool. It has nothing to do with technology tools. You literally go from left to right and value stream maps, there’s a lot written about it. Here’s the thing, Courtney said, you know what, but instead of just every year doing hay or nay, on this COBOL mainframe, why should we get rid of it or should we not. Okay. Let’s wait till next year. Should we get rid of it should or not. You know what? Let’s try something that we’re using very strategically in our web scale or digital properties or our e-commerce, things that we are doing in a really fast moving stop. Let’s apply one of these things to [unintelligible 00:28:23] they did a value stream mapping and they found that it, all the bottom necks were you hand off. And so for years they’ve been debating, wasting time on whether we should get rid of this or not. And they applied a simple technique that’s been around in lean for quite a while called value stream mapping to this mainframe COBOL app. And what they found is, A, the application is fine, the mainframes fine. The cost were buried in the human handoff but they didn’t understand that somebody said this to this person it was always written up because it didn’t have any. And in the end they put note JS application on the forms they got filled out by the requesters and it basically solved 90% of their problems that were perceived problems that were either the mainframe or it was COBOL and we got to get rid of it. So there’s a greater story about like, there is no one size fits all, there is no bimodal, anyway.
Sanjog: Let’s take a quick break listeners, we will be right back. Len, when we come back, we would like to continue with this discussion on what’s that sweet spot for things which are specific to DevOps because we spoke about, okay, fail, fast fail, small and then we spoke about how the culture makes a difference, all of these things are leadership and management 101. So if everything remaining the same, what would you do differently, specifically to get DevOps off the ground and actually move towards the right direction? Please stay tuned listeners, we will be right back.
Sanjog: Welcome back. So Len would love for you to share a few comments that you may have after John spoke about how Nordstrom did what they did. And then subsequently like to have you talk about what’s so unique and different that you’re going to do specifically keeping DevOps in mind versus the standard leadership and management 101 stuff.
Len: John’s comment about, it’s not all about tools and empower power. If a company was strictly going by a bimodal IT kind of theory, they would have certainly bypassed that COBOL as mode one and not even worried about it. I mentioned earlier they were doing a lot of transformation here, so we’re doing a lot of new technology to automate processes and sort of the old school way of looking at this. Especially from the business side is, let’s just throw some new technology against the process that we have and help to solve a problem, right. In John’s example that would have never worked because the technology wasn’t the problem, it’s not about the tools.
Last year I did a presentation at a financial services conference. Title of the presentation was IT is the source of innovation, technology is the easy part and I really believe that. So, what we try to do, is we try to figure out what is the optimal business process. How should I be doing this? How should that COBOL program been interfacing with the people in the other process around it? And then talk about the information that’s going back and forth between all of these things. And then last but not least, talk about how I might automate that with the tool so. Again, I think technology is the easy part here, of course, we’ve got some technical complexities to deal with and we’ve got some choices to make. But the real work here is about around the process in around the people and that applies to– I think both to business problems but to DevOps in general. So you asked me, what have I done that or what am I doing specifically from a DevOps perspective. I think John mentioned this earlier too is, take your small wins when they’re there. So what we’ve decided to do is we’ve broken down into chunks. We’ve made some technology and some tooling decisions. And we’ve implemented those. And we’ve brought in people who can be champions across the additional application development groups that we have. And we call these our lead architects. So we have about 20 or 30 lead architects that represent all of our application developers across the Northern Trust. And they are very, very gung ho about this. So they’re ambassadors. There are strong support in the Application Development Community. And what we’re trying to do with the first chunk of DevOps which is put in and utilize a continuous integration environment, in order to make the developer’s life easier, to increase productivity, to allow them faster feedback and problems in their code and security issues. While we continue to work on the people and process part.
So as we continue to work on, how do we convince the auditors in the regulars, how do we convince the business that this is the right thing, how do we map the fact that we think that there is less risk, as we mentioned earlier. As we’re doing all those things in the background, we’re gaining acceptance for the environment, for the strategy, for the technology component with our application and solution development environment. So we’re taking our small wins, we’re moving forward, we’re gaining incremental progress and we’re building this thing for the long haul.
John: The other thing I think what you are there, Len is what we call learning organization. Which is again– it’s been kind of a cliché over the years [unintelligible 00:34:50] But then again as we narrow it on this thing that DevOps is, again, find the patterns are that companies that– again, there’s no like, you have to start here and do this, you have to do this. You go in and you just continually learn. You guys are learning that CICD is probably working for you. You were learning that developers are getting faster feedback because you’re putting the flow in, faster feedback creates a virtual cycle of success. The faster feedback, the faster effect, the faster you deploy, the faster you get in the hands of the customer. But at the end of day that it is not a pattern that is really evolved harder to measure, I love MTTR and I love Lead Time and I’m a broken record on that. But one of the things that is really interesting in 2016, 17 for me is how can we quantify how good are you as a learning organization. What are the patterns can we actually create a metrics around it. We get tons of information that things you can do, Hackathons, Slacktime, send people to conference, FC is a good example. One of your feedbacks as an employee or– I don’t know what their reward system is but you are expected to teach something to the general community as part of the categories of things that you have to do is part of, I don’t want to say embryos. But objectives of what you need to accomplish. Overall, are you even thinking about it that you are becoming a learning organization? There are tons are really recent literature about this.
Sanjog: So coming to you John, so when you’re looking at these different organizations, at what point can they say, “Okay, we have done enough.” Because to some extent the way that conversation has gone in the kind of things that you’re brought up, both of you, it looks like similar to a continuous improvement wave that we saw some time back. And when we start working on something like that, initially there’s a lot of improvement and lot of investment been made, standards and benchmarks been created. And after some time people started losing steam because you can only squeeze so much juice out of that penny.
John: Well again, I think it’s a journey. There is no ending point that is the famous quote we always say for DevOps. But the DevOps is not an end result, it’s a journey. And it goes back to, steam is about culture. And do you lose steam? Yeah, I mean in certain pockets people are human. But in general, are you an organization that is continually learning. Again, you look at Toyota, I know Toyota had some recent setbacks but for 50 years they basically decimated their market, I mean decimated market. From they got to into late 50s, as they moved into the 70s, from the 70s, it’s almost 2010, no contest. We don’t have enough time or podcast to do the metrics here. But they decimated the automobile manufacturing market. And it was a culture that never lost steam. In fact, one of their steams now they’re starting to predict is management changes that started thinking about ROI over speed and their inherent culture is what actually took them off course. So I say that, again, none of this is easy. Like Len could attribute to this, in a large financial institution, it is really frigging hard. We have a whole chapter in our book that Gene has written most about, he is got the most experience with auditing and audit control and all that, how hard it is but how you can attack things like compliances in different HIPAA or PCCI or all the different things that are – that all the regulations are exactly whatever but the point is, the things that work is, creating a culture that – you can go back to Mike Rother. Mike Rother says that Toyota had a True North. They actually wanted to create one by one flow in their manufacturing. What it is the True North of your organization? And I know some people they’re like, listen, this guy, True North, buzzword, bingo. But it’s not. There’s a reason Toyota decimated the market– for let’s take 70– 40 years. There’s a reason why nobody on the planet can come close to creating video streaming like Netflix. And if you read Reed Hastings Culture Deck, it will blow your mind.
Again, people lose steam for all sorts of reasons but in general you’re trying to create a culture that really does how some form of a True North where everybody gets to see the vision and work that way towards that. I know this is fluffy stuff but it’s not. And it can work and it is working in certain organizations.
Len: I think the first part of that the key word I think that you used and that we’ve been using for this here in Northern Trust is that the journey. I think the key is different people in different roles will in here quotes get it at different times. In our perspective, heavily regulated industry and you mentioned HIPAA industries and financial services. We not only have to get our employees to get it. But we have to get the auditor and the regulators to get it. We have to prove that our new process has the same or better controls in place that the old process. So, you talked about culture. All very, very valid. Northern Trust is a very long tenured staff. I’ve been working here for 33 years. This is my first job out of college and I’ve been doing different technology jobs starting with assembler program to application development to managing applications to, now doing enterprise architecture work. So we have a very long tenured staff with deep technical and business knowledge. We have to ensure that we change the culture in a way that we can keep these partners here, we can incorporate these partners because they’re absolutely vital to our success. We have tons of business knowledge in the application side here at Northern Trust.
With that said, we’re bringing in new recruits, we’re trying to supplement these people, these veterans with fresh resources from computer science universities across the country so. We’re trying to change the culture like you stated John but we’re trying to keep the culture also because we think what we have is valuable in its own right.
John: I’m going to squeeze in here real quick because one thing I don’t want to get confuse is, a couple years ago I did this kind of– I usually do theme presentation throughout the year. And I think it was two years ago I did culture is your strategic weapon. And so that’s a great point and in that– your culture is your thing. Northern Trust or Toyota or Nordstrom, there is a cultural aspect there. So when I say culture, I don’t mean rip and rate, replace. I mean finds the native– what is the vision of this organization. What is made this organization great? I know I sound like a cheerleader now. So what I’m saying is you’re spot on, it isn’t about going into Northern Trust or. Nordstrom or Target and saying get rid of this, get rid of this, get rid of this. There is greatness about an organization. So to figure out what that True North is into to create a journey which actually like embraces that culture as the strategic weapon. So turn that inheriting like– and again, one of the things we’ve got a lot of really cool documented cases where auditors at financial institutions and highly regular bases are embracing DevOps because at the end of the day, you can tell somebody, “Hey, go look at a million lines of code and tell me whether it is– or I can give you some extractions in DevOps tools language.” Like infrastructures code or immutable infrastructure, where I can say, “Will you trust that if we do these things this way that it’s going to be productive?” Will you trust that if the storage people have built their rule sets into the automation? If the IT Sec people who have built, have used cucumber or selenium as part of the continuous delivery pipe why we’re actually not only doing integration and functional testing on the code, you’re doing security and vulnerability analysis on the flow.
Anyway, so it isn’t about – so in my previous one, I want to share that everybody truly understand that I’m not advocating just rip and replace your culture. I’m advocating find out what’s awesome about your culture and take the journey to make it great.
Len: One last comment on that. I completely agree, John. We like to think that here culture isn’t just one thing. It’s a combination of the company’s values, their ethics and their people. And then the other side of that, its part of the culture is the practices and the processes. We at Northern Trust are perfectly happy with the first part, our values, our ethics and our people. What we think we need to do is take a look and change our practices and our processes.
Sanjog: Let’s take a quick break listeners, we will be right back. And Len, I’d like to come back and talk about when we get down to the brass tacks that means you’ve got to put money in there, the restraining and hiring and replacing and repurposing people, all of that is heavy work. So yes, we talk about culture, yes, we talk about all the other things that we have to do anyways. But all of this will take time, energy, effort and dollars. How do you make sure that you’ve got everything like that and to tag onto it? And the other thing is, it really gets ugly before it gets better, whenever you’re trying to do change. Is the organization willing to embrace that ugliness before it will get to the better state where DevOps really starts creating value? Please stay tuned listeners, we will be right back.
Sanjog: Welcome back. Len, let’s talk about brass tacks, getting down to it you need resources and somebody has to fund it and also it could get ugly in the process. What’s the appetite at a business level for that?
Len: In our case, I think that one of the keys here, is you really have to have executive sponsorship for a program like this that goes, high enough in the organization where they can break down silos. They can organize funding, they can explain to other executives what’s trying to be accomplished and what we’re trying to do here and what the goal and the vision is. And I think for Northern Trust that high level sponsor was our CTO. In a large organization like Northern Trust, I think there are a lot of people, especially in the technology side they recognize the issues that we have and be glad to kind of break those apart and tear them down. Like you said, it’s convincing the business of that and we’re trying to do that with sort of the small wins type of thing.
Like I said, we’re starting from transformational programs, we’re looking to improve our delivery, improve the quality of the code and they actually show the business that we’re going to be successful with that. In our perspective, it’s a little bit less about time, money and training than it is about what we talked about before, change management. Culture change, embracing the changes, making sure that we’re putting in place is perceived as being bigger, stronger and faster than what we took down. I think what we’re trying to do is improve our delivery and make the business aware of that. And again, executive sponsorship for a program like this very, very important.
John: Yeah, I couldn’t agree more. Lot of people come to me and say, “John, I work in a large organization and let me explain how it’s breaking up and there is boards that have say, what tools we use.” And I’m like, “I can give you a bunch of do’s to, hacks if I will. But at the end of the day, if you don’t have executive–” I almost feel like should I just tell him to quit or– because you need – I mean at the end of the day, it’s sad sometimes, some of obviously just don’t have that level and it’s often that you do. But let me say this and I don’t want to sound like an anarchist because I do agree, I could not agree with Len, he runs a large business, some large applications that have incredible breath that you don’t change quickly. I worked at a Federal Reserve as a consultant for about a year and a half. There are IMS– there is a tool called IMS that literally runs most of the transactions that run between banks. You don’t change that easy. But the place I will sound like an anarchist is that, there is themes I want to point out, Jamie Dimon, the CEO, JP Morgan Chase, in 2015, letter to shareholders said, “Silicon Valley is coming.” When we talked about lending, when he talked about different areas of that bank or getting attacked by Silicon Valley thinking about people. We hear often enterprises saying don’t get Uber. It is the largest Yellow Cab, I just read last week or week and half ago, largest yellow cab in San Francisco is filing bankruptcy. What is the cost? So you can add up or any biases one of the early cloud evangelists and Edmund Clark used to say, it’s not about bottom ROI, its top line ROI. What is the cost of not doing these things? That’s the thing that – at the end of the day, you can talk about– well, DevOps and training and we’re going to have to bring in new people and change is hard and we’re going to have a bifurcated people who get it versus don’t get, yes, that’s incredibly hard, business is hard, folks, sorry. But at the end of the day, the real threat is – when Jamie Dimon who’s been pretty damn good at his job over the last– I don’t know how many years, sends shareholders a letter says, you need to start thinking about like the threat here. And again, you hear companies over and over this metaphorical hoovering. So again I would tell you that you can pontificate all day long over the reasons why you can’t do this and how hard it is and how much disruption it will cost. And we don’t have a budget this year for this. And I get to say this because I’m not in Len’s role right. But at the end of the day, the real question is whether the cost of not doing these things.
Sanjog: Len, what are the top challenges that the workers or the people who you’re deploying to make all of this happen are sharing? What are they facing?
Len: For the most part, from the technology perspective, so the application developers, the solution architects, the database architects, the scrum masters, the people that are running the agile things. I mean they are our biggest supporters. They are incredibly gung ho and they look at this as sort of a method to modernize their career. So they read the press, they read the blogs, they see the pivotal labs out there and they see, you mentioned Puppet and Andrew Blake, Shaper and all those guys and we try to share some of this information with our constituents, so they’re excited about this. In by enlarge people are excited. I think from a technical perspective the people that are using the products that we’re putting in place, using the processes that we’re laying down. At least at this point in our journey we’re very, very excited about this.
We have what we call Tech Protect, every quarter my group presents about three hours a presentation on a given morning on new things that we’re doing from an architecture and innovation perspective. We have about 500 to 600 hundred people. A couple hundred in the room, several hundred on the phone and these people are incredibly excited about, I think the direction we’re going here and about learning some of this new stuff and becoming more converse in some of these technologies.
Sanjog: 30 seconds John, I have one last question for you. Why do you think would it take for you to be able to sell this DevOps thing to the executive management given, the way it comes across is the geek thing which is within the IT realm.
Len: Yeah. But it’s not a geek thing. We’ve got lot of knowledge out there from Lean, from Toyota pressure systems, if you want to go all the way back to Deming, I’ve done many presentation where I said, called Deming and DevOps. So that knowledge base is that DevOps is really an evolution of Lean, Toyota pressure systems, agile, organizational learning safety culture. We’re actually bringing guys like Sidney Dekker who study human factors and safety. The literature is strong. It’s not a fad. The DevOps, at the end of the day, this is about, I will go back to what I said earlier. It’s about trying to figure out how to get human capital into high performance organizational capital. And I’m telling you there are people now studying that gray area in market caps of companies to try to understand what that is. I will tell you that DevOps is a significant part of that gray area in the market capital of many companies.
Sanjog: On behalf of the show and our listeners, I’d really like to thank you Len and John for sharing your thoughts on how organizations can actually work toward developing a DevOps vision and a roadmap which will take them to success. Thank you so much again.
Len: It’s my pleasure, thank you.
John: Yes, it’s great. Thank you.
Sanjog: Thank you. And listeners, please like us in Facebook, search for CIO Talk Radio and be sure to follow us on Twitter. Thank you again for listening to CIO Talk Radio. This is Sanjog Aul, your talk show host. Till next week, take care and God bless.Less