Mandi Walls: Welcome to Page it to the Limit, a podcast where we explore what it takes to run software in production successfully. We cover leading practices used in the software industry to improve the system reliability and the lives of the people supporting those systems. I’m your host, Mandi Walls. Find me at LNXCHK, on Twitter. Hi folks. Welcome back. This week we’ve got with us Sarah Ryan, PagerDuty. Sarah, welcome to the show. Tell us a bit about what you do and what we’re here to talk about today.
Sarah Ryan: Sure. Thanks for having me. I am in charge of this Center for Excellence of Automation at PagerDuty. I know that’s kind of a mouthful, but basically what I’m doing is looking at ways we can automate various business operations sort of processes within PagerDuty to create some efficiencies in what we’re doing in our day-to-day jobs.
Mandi Walls: That’s awesome. So we’ve had some episodes in the past about IT and technical team automation from that side, but we’re going to talk a bit more about automation for folks who have other roles. So what kinds of teams do you work with to help them out?
Sarah Ryan: So yeah, we can kind of tie that all together too. What I work with is all different teams throughout PagerDuty. So we are also related to some of the DevOps stuff and IT stuff. But the difference between what we do and that is that you can have a human-in-the-loop involved in the process. So it might be an IT process that runs and you need someone to make an approval in the meantime or you need someone to make a decision and that automation will go one way or another way based on what that decision or that approval happens. We can also allow for different if errors occur, error handling, different paths to take place, different logic to happen within the automation. And the other thing I would say that’s different sort of about the standard automation you hear about is that we’re talking about from a low-code, no-code perspective. So you don’t need to be a developer to do it, you don’t need to have any technical experience. It kind of works like a PowerPoint or a Word document where there’s cut and pasting of functions in and you can create the automations on just things you’re doing in your regular job. So doesn’t need to be a large scale automation that you’re doing. It could just be like, “Hey, I’m generating a deck every week that’s taking me three hours to do and this will save me three hours times 52 weeks.” And just adding up all those hours of the various things that you save along the way. So all different departments we’re working with, HR, finance, legal, marketing to name a few. But all of that being said, tying in with some of the stuff you’ve talked about in the past with Runbook Automation where clients that are using Runbook Automation can add on what we’re doing to it and then add on those approvals and the things that might need to happen to change the outcome of what the automation is going to be doing.
Mandi Walls: That’s awesome. So when folks come to you, what are they hoping to get out of it? Is it time savings? Is it more efficient use of their time? Is it error remediation or making sure things don’t go awry? What kinds of things are people looking for?
Sarah Ryan: So I think there’s a few different things. Time savings is kind of the one that people talk about the most, so that’s always there. Also, it’s error handling. So if you’re copying data manually over from Salesforce to a deck or it’s something like there’s a lot of errors that can happen with that, errors in processing POS and things like that, where we’re taking NPO information, someone’s manually entering that into Salesforce and there can be errors involved in that where when you’re doing it with automation and you’re automatically pulling it. You kind of get rid of some of those errors along the way. Also, something we don’t really talk about a lot with automation, but I like to talk about is the sort of quality of life that you’re implementing because these are things that people don’t like to do. They’re a tedious process that you’re doing over and over again. It’s a repetitive process, it’s something that can be automated.
But it’s employee satisfaction, it’s happiness with your job. Something that you know is going to happen the same way every time and it’s something you dread doing because you’re cutting and pasting multiple data points from five different tools. Put that all into one thing, automated, and that makes your job better, makes your day better and allows you to do more creative things and allows you to be more innovative in what you’re doing with your job. You don’t have to worry about, it’s something you can just count on that happening and you not having to spend those hours doing that. So I think that’s my favorite part of automation is just, and I feel like making people’s lives better because they’re not having to do these things that are just the tedious tasks where you’re like, “Oh no, I got to create another report. I got to put this in a PowerPoint deck. I got to put it in a Word document, I got to put it in a Google sheet.”
And sometimes it’s even the same data. You’d be surprised with how often we’re putting the same data on three or four different formats and there’s a need for it. Sometimes you need a summary for an executive, you need a more detailed thing for somebody else. So there is a need for those multiple formats sometimes. But to just not feel like you’re doing that all the time and you’re doing this repetitive tasks and you can be confident it’s accurate, it’s something you take off your mind that you know will be handled by the automation and you don’t have to keep doing it and worrying about the accuracy and worrying about making it happen every week.
Mandi Walls: That’s awesome. On the sort of DevOps and SRE side, we call those tasks toil. That’s just the word that the Google SRE folks have assigned to those things. It’s like, yeah, you have to do it. It’s kind of boring and it can’t be wrong. You have to get in there and actually get it done.
Sarah Ryan: Yeah. And there’s the legal aspect too of doing stuff wrong. Some of our customers that we’ve had, one of our customers we had, had a 25 million fine because they sent out some marketing materials that it was a financial services sort of company, but they sent out some marketing materials that had a decimal in the wrong place. And so they were advertising the wrong information. So they bought our software and they used it in order to implement the automation so they could be certain they would never send out something that had the wrong information.
So that’s another thing I guess I didn’t bring up, but just customer satisfaction where you can deliver more stuff to customers, you can be sure it’s accurate. Another internal automation we’ve done at PagerDuty is a generation of SLA decks for customers. This is something that’s pulling information from five different systems. So it’s coming from PagerDuty, it’s coming from Tableau, it’s coming from Salesforce, a couple other places, and pulling that all into a deck that we generate for customers, for our premium customers. And so a 300 decks get generated and get sent out to the customer. Each one of those decks to would take the person two hours to create. So each one, not a lot, but that’s 600 hours a month that somebody’s creating decks. And to be frank, it wasn’t getting done. There wasn’t enough time to do those 600 decks.
Mandi Walls: You need two FTEs, two or three FTEs just to do that.
Sarah Ryan: Yeah, exactly, exactly. So that’s a level of customer support we can provide that we wouldn’t be able to. Would it make sense to hire a couple people to do that job, but it’s now something we can deliver to our customer that we couldn’t deliver before. So that’s probably the last aspect too, that people don’t talk about as much either. Everybody says the time savings, but really for me it’s the employee satisfaction. It’s like what we can deliver to our customers that we couldn’t deliver before that really adds the value to what you’re doing.
Mandi Walls: That’s super creative as a solution, as something to look at, especially when you’re talking through business applications and some of these things, we don’t think about them as sort of our first targets. For automation, they seem kind of complex and there’s a lot of things going on. So how does a low-code, no-code approach help folks manage and manipulate those kinds of applications for these kinds of solutions?
Sarah Ryan: And I think that’s super important too. So anybody can do it. So anybody can code it, but that’s not even really the important part. The important part is the ideas that come out of all the people. So we’ve done that at this at PagerDuty where we’ve done a couple hack week events. And usually hack week you just think of the engineers participate, developers participate, they come up with some hacks and it’s a popular thing at PagerDuty. I think it’s a fun event. So the hack week we did back in September, I decided let’s get these people automating these system builders who’ve never done a hack week, they don’t know what hack week even is, involved in that. So I started promoting that. And the first one we did, we got maybe about 10 people that build cases, but that one I was just telling you about with the 600 hours a month, that was just someone who that was his job and that was his sort of like, this is a great thing we could do for customers. He came up with that idea as part of hack week and built it on his own and got it done, got it into production, easy. That’s the other thing about no-code, low-code, it’s already all built in. There’s no new server setting up. It’s just all built in. So move it to production, start using it and low time to value. But that’s the important part. It’s all the ideas that you come up with when you branch out to marketing, you branch out to these other departments. These aren’t IT people, they are developers. But when they learn about automation, they think about their own job and they think about, “What am I doing that’s a pain point? What am I doing that’s a repetitive task? What can I automate?” And then you get all these ideas in there that you wouldn’t even think about before because it wouldn’t be something that would occur to a developer, an engineer, it’s not part of what their daily job is. So you kind of draw all those things out. So we did two events, we had the hack week, the PagerDuty hack week was the first one. Then I did an individual event in December where we had 50 people actually that participate in that. So we got 50 ideas out of that. And those two events, it resulted in three and a half million dollars of time savings, just looking at the time savings alone on that. So I think that’s the important thing to think about when you open it up and you say, this is low-code, no-code, anybody can do it, people start learning about it. And as they build one thing and start doing it, they come up with more ideas. So it just kind of spirals out. They tell their friends, they send things to other people and they’re like, “How did you generate this?” And then it just spirals more and more ideas that they get more and more of those efficiencies.
Mandi Walls: It’s inspiring and also it moves the horizon. Like you say, folks weren’t even thinking about things they could have been doing because they’re used to the boundaries of the applications as they’re looking at them. But you added a layer of potential automation or just that manipulation on top of them and it just changes how you interact and what you can do with those applications, it makes them so much more powerful for people and customers and it’s all great stories. And we have another hack week coming up in March, so excited to see what people will do.
Sarah Ryan: Yes. So starting to promote that a little bit amongst the automation duty people that are in my Slack channel. I’ll just start thinking of their ideas and get that going. So yeah, it’s fun to get all those ideas. And even when we go to customers that are using it, they always have their own ideas too. So we have typical use cases that people do that publish and we talk about, but then there’s always people coming up with more ideas and it’s kind of making the world more efficient, coming up with ideas to do your jobs better, make the world more efficient and save time and money and risk and having us and all that.
Mandi Walls: Since there are so many potential things that people could be looking at, do you have things that you suggest people look at first? Do you apply to things that are easy or things that would save the most time? Or is there any kind of rubric that you use with folks to help them sort of focus?
Sarah Ryan: Yeah. We do use a rubric where we’re looking at complexity and value. I think the mistake you make at first is you’re looking for the highest value thing out there. And that’s not always the best thing because sometimes that high value thing can be super complicated. So what we always recommend starting off with is the low complexity stuff. If you can find something that’s high value, low complexity, great. But the most important thing is the low complexity because then you can get your feet wet, especially when you’re talking about no-code, low-code building, something that’s low complexity. Maybe it’s just an email you’re sending on every week with a status report to your team and it’s saving you couple hours a week and that’s a little bit of time, but it gets your feet wet, and trying to figure out how the whole system works and how to build that and then you can move forward from there and go to more high complexity things as you move on. But also, when you’re identifying use cases, what I always say is, “What’s your pain point? What is the biggest pain point in your job? And then let’s see what we can do to automate it.” Because the automation, the way it’s set up, it’s kind of like a Swiss Army knife. You can do almost anything. You’re only limited by your imagination. We have built-in integrations to a lot of systems out there, but if a system has an API, we can call that API and we can integrate it that way as well. So what I always say is, what’s your biggest pain point, your most tedious thing, or even something that’s not working as a process. So we talk about also process redesign, and people say, “Well, why would you automate a band process?” Well, sometimes you can’t fix the band process. The ideal thing is to fix the process, then do the automations, but you can’t fix the process. A company like PagerDuty or even bigger companies, some processes are just really ingrained into what everybody’s doing and you can’t fix it. So even if we can take out a portion of the process that maybe it’s a 12-week procurement process that you have to go through, if you can take out some of those approvals, do some parallel approvals within procurement and get that to move faster, that might not be saving necessarily a lot of time because you’re still doing the same level of approvals and you’re still going through the same process. But you’re allowing people to do parallel approvals to review documents at the same time, store that information and you’re just moving the timeline a little faster. So just kind of taking away that pain point of, “I need to procure this. I’m waiting 12 weeks, 16 weeks, I can’t do my job without getting some work procurement handled.” So that’s not a dollar amount of value and time save, but that’s a pain point of something that needs to be fixed and might be continuing with the same process. It’s not the most efficient process, but we’re helping that process along and we’re taking at least the people, the pain of the people involved in that process out of it and letting a bot do that automation.
Mandi Walls: Absolutely. I feel like it helps people just feel like they’ve been heard. Yeah, we’re going to work on improving this horrible process, whatever it happens to be.
Sarah Ryan: Yeah. And then you can slowly, you automate parts of it, but you can slowly take out pieces of it too. And you’d be like, okay, maybe we don’t need seven approvals. We’re automating these seven approvals, but maybe we can do it with four approvals. I know this was a case with some of the HR because we also did an HR onboarding process and we looked and there was something like 37 emails going out as part of the onboarding process. And so just calculate that, you’re like, “Okay, I’m spending 10 minutes in email, 37 emails going out to this many new hires.” Lot of time. But also some of the emails we were sending out, HR would send out an email and then Mindtickle would send out the same email. And as we started digging into it, I was like, “Well, are these emails saying anything different?” They’re like, “No.” I’m like, “Well, does the person need to receive a Mindtickle email and an HR email? If they do, fine, we can do that.” But that allows you to look at the process overall and say, “No, we don’t. We don’t need to send out three emails saying the same thing at the same time and just coming from a different from address.” So I think that process redesign naturally happens when you’re automating, but there’s always that debate. In an ideal world, you go in and redesign and make all your processes as efficient as you can and then automate them to make them more efficient. But that’s just not going to happen. People just don’t have the time to do that. So just automating even what you cut along the way really I think helps people along to make those processes more easy.
Mandi Walls: Yeah, absolutely. And it shines some light on where those dark corners are in some of those things and hopefully makes people maybe ask the question, “Why are we doing this, this way?” And it’s just an opportunity to reevaluate anyway.
Sarah Ryan: Right. And that’s even true too sometimes with some of the systems that we’re integrating with where people are like, “I need to pull this information out here and do this.” And they’re like, “Oh, we can just create that report in Salesforce for you. We can just create that Tableau report. We don’t need to.” And so just even shining the light out, “Here’s what we’re doing, here’s the process of what’s going on.” And then sometimes it comes back where the person who’s the admin of that tool might say, “Well, you don’t need to automate that. You can do that within the tool itself.” And that allows us to even expand what we’re doing with the tools we already have and not necessarily go through the automation but highlight, here’s some additional features we could be using that we’re not using, that we’re paying for with these tools.
Mandi Walls: Absolutely. We have so many tools, I don’t know how you’d ever know all the features in all the tools is like you open up Okta and you’re like, “Oh my god, there’s like 80 things here and I don’t even know what half of them are, let alone how to use the other half that we have.”
Sarah Ryan: Yes. For sure.
Mandi Walls: So much stuff. One question that we ask folks occasionally is, is there a common myth or misconception about the stuff that you do, business process automation or even low-code, no-code things, anything that folks get wrong pretty often that we can set the record straight here for?
Sarah Ryan: Yeah, I think a couple things. I think the first thing is that we’ve talked about before, but that you had to be developer to build an automation. You don’t for sure. That’s the whole point of no-code, low-code. So that’s the primary thing I would say. But then also there’s automation tool. There’s a lot of automation tools out there in the marketplace. So some tools are RPA tools where they’re just running a process and they’re doing it the same way and they might have a little bit of logic that can be built in but aren’t the full scope of what we can do, where what we’re really doing is orchestrating a whole workflow. So we’re orchestrating a whole process. So there’s people in the loop, there’s different paths you could take with this automation. There might be a portion of it that just is running a batch or calling something that’s running a batch. But after that batch is run, different people get notified, different approvals happen. So it’s really the automation that we do and the tools that we now have at PagerDuty with this workflow automation has really orchestrating that whole level of automation, not just one small portion. Integrating with everything creating, we can create an instant and PagerDuty, we can update Salesforce, we can do all that in one flow and it’s not a disparate process. So I think that’s the biggest thing is just when you look at all those tools out there to know what you can do with our tools versus just like an RPA tool out there that many customers we have may have an RPA tool. Well, we can also integrate with that RPA tool. We can make a call to the RPA tool and run the batch that’s already been set up in there so you don’t have to redo what you’ve already done with other automation tools when you’re using our tools.
Mandi Walls: So you came into PagerDuty when we acquired Catalytic, which was last year?
Sarah Ryan: Right, last April. So it’s almost been a year.
Mandi Walls: How much of the stuff is available for folks right now externally that they could play with or get started?
Sarah Ryan: Currently, we have not been selling externally. We have about 30 customers that are running on Catalytic that were existing customers that came over in the acquisition. So those customers are still continuing to use Catalytic, they’re using it in production, they’re using it for many different things. And now this year we’re just starting to start selling Catalytic to external customers. So that’s a new thing. We’re still working on the process for that. But we’ve had a lot of customers asking us. So particularly part of process automation deals where they say, “We need to be able to do this. We need to put a web form on the front of it, where someone can fill out information, then run the automation.” So we’ve had a lot of customers asking about it when we’re selling other things to them. So now, because of all the demand we’ve been hearing out there in the market, now we’re starting to open up more where we can start selling it to customers out there as well.
Mandi Walls: That’s awesome.
Sarah Ryan: Yeah. So that’ll be cool. And eventually it’ll go into the flexible platform that is being built that eventually will be upgraded. So the existing people using Catalytic and using that tool on the workflow automation that we’re going to begin selling to other customers that’ll eventually be upgraded to the new platform that we release next year.
Mandi Walls: It’s going to be so powerful for people just to pull in all of this stuff, get all of their information from all these sequestered systems that are powerful individually. But because all the information is hidden in all these silos, you feel like you don’t get the full picture. And especially when you are responding to incidents and working in unplanned work, we want all the rest of the information. We don’t want to have it hidden behind other logins or other servers or other systems that we don’t all have access to. So this is going to be stupid powerful.
Sarah Ryan: Yeah. Even talking about postmortems and things like that, people are doing stuff, creating automated documents from all the information that happened within an incident, and then being able to send out that postmortem summary within one document that everybody can see and it can go to the appropriate people through PagerDuty. Whatever is set up as the escalation policies and PagerDuty, it will happen. But that automation of consolidating all the information from the various systems that are needed will be able to be done through the automation with this tool. So yeah, it’s really cool. All the things are getting done out there. And again, it’s considered you’re only limited by your imagination of what can be done.
Mandi Walls: Yeah. Anything that’s in your workflow because we don’t know what folks are out there doing and yeah, you plug in any number of different systems together, who knows what you’re going to get out the other side.
Sarah Ryan: Yeah. People are using it in HR for recruiting purposes where you have the [inaudible 00:21:57] information, scheduling the interviews, checking schedules, scheduling the Zoom call, being able to pull the information of who attended Zoom calls, things like that, all can happen in that. So yeah, there’s been a lot of interesting ideas that have come out through PagerDuty and a lot of interesting stuff our customers are doing as well. And we’re thrilled the ways they’re finding to use the product and the ways that we can help them.
Mandi Walls: So as you’ve been working with folks on this in the last few years, is there something that is sort of a lesson learned or something you feel like folks should want to know as they’re getting into this kind of work or looking for this kind of solution?
Sarah Ryan: Yeah. And I think we kind of touched on this before a little bit, but I think the biggest thing is just to start with the lower complexity thing and don’t necessarily look at dollars, but look at value in other ways. Not always the time savings necessarily. Time savings are great, but start with the low complexity stuff, get the ideas. The other thing I think sometimes we’ve made a mistake in the past is focusing on one department. So we’ve had customers where we’ve only signed with the marketing department. And so there was a license for only marketing to use it. It needs to spread across other departments, get all the ideas that are out there and then you can really get the full value out of it. You don’t get the full value out of it if you only are looking at one department and only getting ideas from one place. So I think it’s low cost to sort of set that up. And we talk about the hack weeks. Even for our customers, doing a hack week, having people come up with their ideas, come back with their ideas and build that. Our hack week stuff we’ve done in the past has been about 10,000 hours we’ve saved from each hack week. So it’s like we’ve found automations that have saved 10,000 hours of people time with one week of work.
Mandi Walls: That’s crazy. Oh my gosh. Wow.
Sarah Ryan: Yeah. And that just spirals out because all those people who had those ideas share with their friends and they come up with more ideas and then it just gets bigger. You think they’re going to run out of ideas, but there’s no shortage of ideas out there and there’s no shortage of ways to be more efficient. So you really never run out of ideas. And the customers that really use our product really get value out of it because they’re paying us a fee and then they keep building more and more stuff and getting more and more efficiencies out of it so they really get their money’s worth.
Mandi Walls: That’s excellent. This is so exciting. I just can’t wait for all this stuff to be ready to go for everybody. And so everyone gets all of the benefits and enhancements and gets all the time back from doing all these manual things that we have to do and all this weird scraping data from one place and putting it into another to get all the value out of it.
Sarah Ryan: Yeah. And I love this just getting the word out there because again, we have customers already using this. This is a production tool that many customers are using. So it is ready, it’s going to only get better and better as time goes on and we make improvements and enhancements and stuff to it. But just getting the word out there and letting people know about it and getting people interested where they can learn more and come up with their own ideas of things they can do is really always been my goal and everything. So I’m happy that this is sort of another vehicle to just get the word out there and happy to have people reach out to me with any questions and I can help anybody learn how to automate or… I always also say everybody has an idea.
Mandi Walls: Yes.
Sarah Ryan: Everybody has something they could be doing more efficiently or they could be doing better. So anybody who needs to brainstorm an idea, I can help with that too as well.
Mandi Walls: That’s great. So to wrap up, where can people find you? Where do you hang out?
Sarah Ryan: So we have a Slack channel automation duty COE that you can go to and join that. And there’s a lot of activity in there, a lot of people.
Mandi Walls: That’s just internal to PagerDuty though.
Sarah Ryan: Internal to PagerDuty, yes. Yep.
Mandi Walls: Okay. Do you guys participate in our community forums at all if folks have questions or that stuff?
Sarah Ryan: No, we haven’t done that yet today. But that’s a good thing I should look into for sure.
Mandi Walls: All right. Yeah, we’ll have to… As folks get more interested in this and things they want to do, we’ll get them hooked up over there.
Sarah Ryan: And I would say if external customers are interested, contact your current PagerDuty rep and they can get you in contact with us to figure out how you can do more automation.
Mandi Walls: That’d be excellent. We totally want to hear from everybody out there who has automation ideas, things they want to plug into all these amazing platforms.
Sarah Ryan: Yeah. And we’re beginning to build some kind of customer stories and things like that, they’ll be out able to share out as well. So some of the more communication things will be happening this year too, where we’ll be crafting some more of those stories to be able to tell to customers as well.
Mandi Walls: Well, excellent. I’m looking forward to all of this. Like I said, we’ve got an internal hack week coming up, so that’ll be super fun to see what else folks come up with for all of this. And yeah, like you say, if anyone has questions or you’re going to reach out to your account team, you can do that. If you do have a question, not sure who to get in touch with, you can join our community forums and we’ll try and get you in the right direction there. And that’s just community.pagerduty.com. It’s also in the show notes for this episode. So thank you so much for bringing us all this exciting stuff.
Sarah Ryan: Of course. Yeah, happy to be here. Thank you so much for having me.
Mandi Walls: That’s excellent. So yeah, like I said, for anybody who has any additional questions, check us out on the community forums. We’ll be back in a couple of weeks with another episode. And in the meantime, we’ll wish you an uneventful day. That does it for another installment of Page it to the Limit. We’d like to thank our sponsor, PagerDuty, for making this podcast possible. Remember to subscribe to this podcast if you like what you’ve heard. You can find our show notes at pageittothelimit.com and you can read us on Twitter, @Pageit2thelimit, using the number, 2. Thank you so much for joining us, and remember, uneventful days are beautiful days.
Sarah works for PagerDuty as the lead of the Center of Excellence for Automation at Pager Duty working on all things automation to create more operational efficiencies at Pager Duty. One of her favorite things to do is talk to people about how they can save time and money by automating the tedious tasks that they need to perform in their jobs on a regular basis thus not only saving time, but many headaches for her fellow Dutonians. She lives in Chicago with her family including three kids and many pets!
Mandi Walls is a DevOps Advocate at PagerDuty. For PagerDuty, she helps organizations along their IT Modernization journey. Prior to PagerDuty, she worked at Chef Software and AOL. She is an international speaker on DevOps topics and the author of the whitepaper “Building A DevOps Culture”, published by O’Reilly.