Episode 117: PASS Summit Retrospective

Have you ever had a week without a Wednesday?  Me neither, so for this episode, we recap our experience at PASS Summit and talk about all the amazing compañeros we met and give some thoughts around our very first meetup.  It was so much fun to meet everyone and both Steve and I gave sessions, which was a great experience–and a first for me.

One, we both realized after the fact is, we are not good at social media or taking pictures!  We apologize we don’t have a bit more to share on that front, but hey–we do have faces for radio.

Episode Quotes

“I find when I’m at a conference I get so busy and I enjoy talking with people. I just don’t think to take out the cameras down for shot.”

“It’s nice to have live music but at the same time it made a little difficult to hear sometimes.”

“Watching these keynotes from the hotel is super convenient.”

“Good questions that’s always make you feel good as a speaker.”

“Be prepared, bring two laptops and a backup plan, and a backup to your backup plan.”

“The last 5 minutes before a session starts, even if everything is ready to go and going smoothly, the last 5 minutes is the stress factor.”


Listen to Learn

02:32 Companero Shoutouts

07:47 SQL Server in the News

10:50 PASS Summit Retrospective, being on social media

12:47 Tuesday night at Seattle – Carlos’ Mardi Gras suit, the live music

15:56 Volunteer Party18:46 Thoughts about the livestreaming of keynotes

20:13 Wednesday sessions, SQL Trail Mix event

25:10 Thursday – Steve’s session on Database Corruption

31:15 Thoughts about the Game Night

32:29 The Sponsor Area

34:07 Friday – Steve’s session (Backup and Corruption) and Carlos’ session (Establishing Baselines)

40:23 End of conference and unfortunate story at dinner

Transcription: The PASS Summit Retrospective

Carlos: Companeros! It’s nice to have you on the SQL Trail again.

Steve: Yes, and welcome to Episode 117. This is our PASS Summit retrospective episode, which I’ll tell you we got to meet a lot of companeros last week.

Carlos: Yes, and it was great meeting a lot of you and getting to chat with you. I feel like I’ve been doing a little bit of retrospective, a couple of retrospectives here. It is the conference season so we’re kind of giving our thoughts around the conference, what went well and what we enjoyed being out there in Seattle.

Steve: And I know I say last week, but yeah, it was last week when we’re recording this but it will be two weeks back when it actually airs.

Carlos: That’s right. Man, how time flies.

Steve: Yeah, and I guess with you being on the East Coast and me being on the West Coast we don’t have that many opportunities where we’re actually physically in the same location.

Carlos: That’s right. And we did, again you’ll see as we kind of get into this just some of the activities that we had. But our schedule was jam-packed and some client issues kind of rose and we didn’t have as much time as I was hoping. So for example we didn’t get to do an episode, I don’t want to say live, but face to face, which has been something that we thought about but didn’t plan well. But this works and hopefully the companeros will enjoy it.

Steve: Maybe next time we’ll have to have you come a day earlier or something just to do that. Or I’ll go a day early depending on where we’re going to.

Carlos: Right, there you go. There is always next year, right, sound like a bad sports team. Ok, so getting into some companero shoutouts, and because of Summit and running into so many of you. We’re actually going to break this up a little bit, so if we don’t mention your name now don’t freak out just yet because we may be working some of that into the actual story or our retrospective. So first we have Justin Wailey, who I think I incorrectly identified as being from Georgia. His Twitter account says he is in Alabama. It was good to connect with Justin. I had met him two years ago. I think it was his first Summit back then and it was good to reconnect with him.

Steve: Yup, and we’ve got Tom Norman on the list too. I don’t remember meeting Tom, do you?

Carlos: Yes. He was on our Trail Mix, but I did run into him at the opening social. He was a buddy and so we chatted. Tom is here from the East Coast and it was nice to connect with him again. Now another one that we both talked to kind of in sessions so actually in the convention center, and it’s Brent Unruh. I’m pretty sure I’m pronouncing that right. He told me how to pronounce it. Hopefully I’m getting that right Brent. But BrentU on Twitter and he is the one who brought us the question because many people talked about flying and his question was, “Would you still choose flying if you can only go 3 miles an hour?”

Steve: Oh yeah, that was long time back on the podcast, SQL Family session wasn’t it?

Carlos: That’s right. So you’re long time listener Brent. Thanks for reaching out and connecting with us.

Steve: Yup, and then we also talked with Andy Levy from Rochester.

Carlos: Yes, and we’ll probably bring him back up maybe when we get to the sessions but nice to meet him. Contributor on the DBA Tools team and it was nice to connect with him.

Steve: Yup, and I know he and I have been back and forth on LinkedIn a little bit since the conference talking about Database Health Monitor.

Carlos: Oh, got you. Ok. Another DBA Tools regular is Constantine Kokkinos. It’ was good to meet him. And he is the one who got me the, well I guess so Andy had them. He was the one who brought them but I love the DBA Tools.

Steve: You know, there were quite a few members of the DBA Tools group there and it was good to meet Constantine.

Carlos: Yes. I don’t think Claudio was there but I did talked with Rob. Unfortunately I did not meet Chrissy, wanted to do that but for a reason didn’t happen.

Steve: Oh! Yeah, I got to meet Chrissy in the speaker room. Sort of they are practicing. I heard her talking and I did not know what she looked like but I just totally recognized her voice from the podcast.

Carlos: Oh, got you. Ok.

Steve: So I got to say hi and met her there as well.

Carlos: Oh very nice. And then a couple of our MVPs want to give shoutouts to Argenis Fernandez, Brent Ozar, Cathy Kellenberger, so thanks for coming out to the Trail Mix event and talking with us, giving some of your thoughts. I know everyone appreciate getting your thoughts on some things and mingling with us so we do appreciate that.

Steve: Yeah, it was an interesting group to get to know there.

Carlos: Sean Barry is another one. Good to meet him. Kevin Wilkes, long time supporter and previous guest on the show. Adam Jacobson who is actually someone who have introduced us to a client and happen to be there, so it was nice to be able to catch up with him.

Steve: Yup, I did not get to talk to him as much as I would like to but we did talk briefly.

Carlos: Yes, and that was a reoccurring theme I think for Summit is, and that was a surprise. I mean, particularly with the podcast and that’s really where and at the SQL Saturdays I feel I want to be connected. That’s one of the things that I want people to know me for if you will is being connected. And I was surprised to just how many people I wasn’t able to talked to, so I got like empathic, Oh yeah, I got to follow up with them and ended up, I’m at home.

Steve: Well, you know, on that point. I mean, you talk to some people and they think, “Oh, you’re going to a conference for the week. It’s like a vacation. Time to relax.” But, holy cow, it was busy. I mean, one, it never stopped it seem like which is awesome. I mean, I really enjoyed it but it’s certainly no vacation. It’s busy.

Carlos: That’s right, that’s right, and then from our perspective as well we are trying to, like everyone. Not saying that it’s unique to us but trying to understand the ramifications of all the new features that Microsoft is dumping out there on us, the keynotes. We had that keynote on the Cosmos DB. That’s something that I think all of us are trying to understand, ok, what’s our future is going to be like. What’s all looking like and understanding some of those concepts. That takes brain cycles as well.

Steve: And speaking of that future, I mean does that bring us right into SQL Server News?

Carlos: It does. Yes, let’s do the SQL Server in the News. So we ran into Travis, and Travis he’s the PM for the Linux migration and I’m not exactly sure how his role has evolve. I think it’s still something that he is still doing. But we were chatting and of course they had mentioned many of the new features. We won’t cover all of them today. We’ll save some of those and kind of spread them out for future episodes. What he mentioned to me and he said that they may be announcing everything that they have just put out and SQL Server obviously became live at the beginning of the month. They are going to announce VNext as potentially as early as next week.

Steve: Wow! So if you’re not on SQL Server 2017 yet you will soon be behind.

Carlos: Yeah, that’s right. You just made to 2018.

Steve: VNext will be taking over in the near future perhaps 2018.

Carlos: Yeah, so again, just kind of continuing that cycle. And you used to think about the platform going on to Linux and all the additional features that they are trying to integrate there and what the monumental challenge that is.

Steve: Oh yeah. It’s really interesting because part of it is there are so many new things going into SQL Server at the rate that it’s going out right now and other Azure related databases and things. I mean, it’s really tough to figure out, ok what are the things that are really going to stick and are going to be the fundamental pieces that we’re going to need to know going forward, and what are the things that are maybe a little bit of hype and maybe don’t end up being that big of a deal.

Carlos: You know that’s right. I think the comment is we don’t need to feel. And even us as consultants I feel like we’re in a pretty good position because we get to be lots of different environments and touch of this new technology sometimes. But a lot of times I can feel like I’m in the Stone Age because all the new things that people are trying to do. You know, cloud this and cloud that. I think companeros you don’t need to be afraid if you’re not on VNext. Don’t despair.

Steve: Right. And I think, but that’s part of the benefit of the conference too is you go and you’re exposed to a lot of this new information. And it sort of pushes you out of that comfort zone that you’ve been in for the last couple of years perhaps.

Carlos: Sure, exactly. And if nothing more like to be educated how things are changing and the types of problems that people are trying to solve. Your current organization may be not trying to solve that problem but if you find a problem that you’re then interested in again you’re creating value and then trying to go out and solve that for a different place or a different organization. Who knows what will happen, but being in a position to be able to do that is valuable.

Steve: Yeah, absolutely.

Carlos: Ok, so ultimately today it is Steve and I are doing the retrospective, we will have some shownotes. Now, I would say pictures and I guess we can get into this a little bit but we as busy as we were you could tell that we’re not great social media people, and do not take nearly enough pictures.

Steve: Yup, and you know, that’s the thing I find when I’m at a conference I get so busy and I enjoy talking with people. I just don’t think to take out the cameras down for shot where like there are others who say, “Oh look, I got a picture with this person with that person.” Wow, I just didn’t have time to do that. I think I need to make time for that next time.

Carlos: I think we are showing our age there a little bit, right? We’re not the selfie first kind of people.

Steve: Yeah.

Carlos: And that’s like for Twitter. I mean, I like to be on Twitter and put some things out there but I’m definitely not one of those people that has to put my every waking moment detail on there. And then you get busy if that’s not the habit, right? You haven’t trained yourself to do that then the day goes by and you’re like, “Opps.” In fact today I was actually replying to some tweets that happened during the Summit, so that was kind of embarrassing.

Steve: Yeah, it’s like you need an extra 4 hours of each night to go back and catch up on everything.

Carlos: That’s right. My old man disease is already kicking in pretty good. Particularly in the West Coast time, right, I’m there in Seattle or being that my body is on East Coast time rather. I’m in Seattle, I’m ready to go to bed. But the shownotes episodes are going to be at sqldatapartners.com/summit.

Steve: Or at sqldatapartners.com/117 for the episode number.

Carlos: Yeah, so as far as some of the other things that we did. So Tuesday was kind low key, actually we’re in Seattle. I know that there was like the user group meeting or the chatgroup leader meetings or whatnot. We actually had some client engagement meetings so we took advantage some of that time particularly with me being on the East Coast to do a little bit of that. But we didn’t get to check in and I started meeting people.

Steve: Yup, and on Tuesday night there was the Welcome Reception for everyone, and we went to that. That was a lot of fun and got to see a number of people that I hadn’t seen in a year or so.

Carlos: Yes, and if you missed me, I’m sorry I was trying to stand out as much as possible in the Mardi Gras suit.

Steve: Right, some people might have thought you were an entertainer of sometime with that suit Carlos rather than actually an attendee.

Carlos: Yes. I was accused more than ones, “Are you hired to…” “Nope. Just me.”

Steve: Well, that was a pretty awesome suit and if you haven’t seen Carlos in his I guess Mardi Gras suit we could call it?

Carlos: Yes, that’s right.

Steve: Yeah, take a look online and there are some pictures of him on that.

Carlos: Yes, we actually get those up on social media. Now, one thing I thought was interesting and I don’t remember. Now, I missed last year’s Summit. I don’t remember them doing live music at the welcome.

Steve: No, that was different because last year I think there was not a live band last year. It was recorded music and they were doing like slideshow or something like that I think last year.

Carlos: So I thought it’s nice to have live music but at the same time it made a little difficult to hear sometimes.

Steve: Right, and we are standing chatting with someone and you realized that it’s so loud you’ve got to be six inches from their face just to hear them. It makes it tough to communicate. If I had any say in it, I would love to say, maybe we’re getting old, but turn down the music. That sounds like an old man story there.

Carlos: Yeah, I really have to rename this “The Old Man Review”.

Steve: I guess we’re showing through.

Carlos: I agree, it’s one of those things. I don’t know if we could have a little bit more time. Of course people come in and out of that as well, but maybe pausing that music or giving us an hour before it plays. I don’t think they did take a rest, but yeah, I don’t know.

Steve: And of course, Tuesday night was the Volunteer Party.

Carlos: Yes, and so again more old man. I stand at the welcome reception a little longer than you did. I know you and Derek went over to that. I did not want to walk over there. I guess I was too cheap to take an Uber. Oh and that was when Andy Leonard actually gave me a book. So I had this book that I’m like, “What the heck I’m going to do with that?” So I ended up taking it back to the hotel room to drop it off and then deciding, “No, I’m not going to make it out there.”

Steve: Yup. Well, so I ended up going to the volunteer reception or volunteer party I guess. What I like about it is a couple of things. One, free food. So getting to meet some of the other speakers, some of the other PASS chapter leaders, some of the other volunteers, I mean, it’s a great group of people. What I would say is sign up and volunteer in some way if you’re not part of that because then you could be part of it.

Carlos: Yeah, that’s right. And you meet the Costa Rican guys, right? The Hispanic guys?

Steve: Yeah. I guess there were 4 or 5 of them and I think I ran into them that same party last year. Yeah, good group of guys. Bright guys from Costa Rica.

Carlos: Yeah, good times. And I did the mistake of then trying to go out afterwards because the DBA Tools Team had like a reception I saw on Twitter. I’m like, “Oh, that’s just right here. I’ll go the block.” When I got there they were actually all headed to SQL Karaoke which I then felt stupid because I just got in there and it was ending. Here I didn’t want to walk to the volunteer thing so then I ended up walking all the way to SQL Karaoke and if you know where that is. It’s like on the borders of Seattle, so I walked a lot more than that what I wanted to that evening. But it was all good, so one of the guys I got to meet in line actually. Again, feeling stupid again because I didn’t hit to that ticket to get into SQL Karaoke so there was that whole thing. But the SQLBits, no not SQLBits. SQLBits is Rodney from Florida. They guy who wrote the SQL Server 2016 song.

Steve: Oh yeah. I forgot his name but he was at SQL Saturday at Redmond last year.

Carlos: Right, so he was in line there behind me and so we started chatting and that was a lot of fun.

Steve: And he was going to Karaoke even though he is a real musician.

Carlos: That’s right. But we ended up, “Ok let me see. I’ll show you how this is done.”
I also ran into, so in the opening reception another name, Bert Wagner. It was funny because I walked up to Bert and like. I know you, I can’t remember where I know you but I feel like we’ve met. And then it wasn’t until Wednesday he actually came to our SQL Trail event and like, “Ah! Now I remember from the sign up list.”

Steve: Oh yeah. And I remember meeting Bert as well and talking a little bit about his job and what he does. Yeah, definitely cool.

Carlos: Right. So one of the takeaways and one of the things now, I’m not sure that we’re maybe promoting this idea but we found that watching these keynotes from the hotel is super convenient. Now, that does mean that you’re kind of miss out on breakfast where you have to kind of go on your own way for breakfast. But that’s something that we did and just worked out really well.

Steve: Oh yeah. And what I liked about that was because normally you go to the keynote and it’s jam-packed full, and it’s hard to see. I mean, depending on where you’re sitting in it could be a challenge. So just watching from the live broadcast and watching from the hotel room then you’ve got the chance while you’re doing that when they mention something then go look it up. Go online and take a look at it while you’re watching it. And I think that I like doing it that way rather than being there where you’re just sort of stuck in a seat and glued to the room.

Carlos: Right. I’m not sure if it was my tablet or what. mood of your laptop. It seem to be fine but you do sometimes hit that streaming issues. But once they got it going, PASS did a good job on their side to at least make sure that plenty of streams were available.

Steve: Yup. So then Wednesday there were some great sessions and I know I got to see a couple of those. Then later in the day on Wednesday we had the SQL Trail Mix event.

Carlos: Yes, we weren’t quite sure what to make of this, right? How this is going to turn out and special thanks here to Sean and Jen McAllen. We actually reached out to them to get their thoughts they’ve been doing several events there at Summit. And we thought, ok, because you do hear about all these different events would there be room enough for ours.

Steve: Would people want to go to ours?

Carlos: Would people come? That’s right. So it’s Steve and I sitting there eating peanuts, so these are all kind of thoughts kind of growing through our head. So we said, “Ok, let’s do it. And we’ll reach out obviously to the companeros and to our podcast guests and we’ll kind of go from there. I think we were pleased with the turnout. Who was there, thank you, so if you were there companeros it was great to meet you. I guess there are a couple that we want to specifically point out here but we had a great time chatting with you.

Steve: And I think what really surprised me was the small amount of promotion that we did, the small amount of time we had to promote it, and how many people actually showed up in that small amount of promotion time.

Carlos: Right, so that is true. If you didn’t get an invite for whatever reason, so we apologize because we did fill up way faster than we thought it would. So again, we were kind of thinking 25 people would be super successful. That’s was kind of our number. We knew some people wouldn’t come. We ended cutting off registration at 45 and that happened in a matter of about three hours, so it was super quick.

Steve: Yup, and I guess just a little bit of background for those who don’t know that the SQL Trail Mix was. This was an event that we hosted where it was a restaurant/bar where we reserved some space. We got a number of tables and we bought appetizers for everyone although we didn’t buy drinks for everyone. Everyone was on the hook for their own drinks. And it was just show up and chat. Show up and meet other podcast listeners. Meet other podcast guests and give us feedback on the podcast. I had a great time. It was awesome.

Carlos: I agree. It was good. For example, rubbing shoulders with James, so James Youkhanis He has been on Twitter. We’ve exchanged emails. He has suggested several podcasts topics and so to finally been able to meet him, rub shoulders with him was actually pretty cool. The guys from Connecticut, Will Conklin and Chris Albert, long time listeners of the podcast, so we shoutout to you guys. Thanks for coming up and chatting with us. And then Josh Simar. I can’t remember all of a sudden where Josh is from, but he had mentioned that he’s been listening to the podcast and it was great to chat with him as well.

Steve: And then also, Andre Ranieri is a friend of mine that I’ve known through PASS events for quite a while and he showed up at the event and have a chance to catch him up a little bit.

Carlos: Right, and that was an interesting thing. While we had our registration, I don’t know, we probably ended up with 30-35 people, because people did come in and out. Registration kind of help but it didn’t indicate everyone so I guess there is some thought we need to put into how to do that. We just want to make sure that we could provide for everyone who did show up.

Steve: Yup, and I think if we’re going to do it again or I hope we do it again. I think maybe going a little bit bigger but not massive would be good.

Carlos: What was interesting because I think kind of worked to our favor potentially is that we didn’t realized that we’d also be going up against seven world series game. I think that impacted just the bar itself because when we talk, “Yeah, out Wednesdays are dead. Why don’t you come it would be great to have a group.” And then we get in there and I’m like this is dead.

Steve: Oh yeah, it was packed. It really was.

Carlos: Ultimately we would very much like to continue to meet up with you guys and I’m not sure how well this would work at other events but if we have opportunities to something like this of course we’ll try to do it again.

Steve: So then that brings us to Thursday.

Carlos: Yes, that’s when Summit got real in the sense because you had your session. You had the first of two sessions.

Steve: First of two sessions. This was the first time at PASS Summit that I actually had two sessions to present. The first one was on Database Corruption, and I was on early afternoon I think. It was one of those things where as a speaker you always worry about getting connected, at least I do. Is the presentation going to work? Is the laptop going to work and all that? The laptop I’ve had for about 2½ I’ve never had a problem. Everywhere I’ve gone, you plug it in, it work just fine. It’s even work fine at two former PASS Summits on similar equipment. But still every time we go I tried out. There is this speaker ready room that they have to try it out on their projector and make sure it works. Try that all out ahead of time and get there and it will not for some reason the multi-video output will not connect to the projector. And of course, I’m the kind of guy when I present, that as soon as the previous speaker is off the stage I am there taking every minute I can to make sure that it’s going to work. And it didn’t work, and it didn’t work.

Carlos: And the AV people came up and they were trying to help you.

Steve: Yup and some of our friends Derek Bovenkamp and Randolph West. I mean they were in the audience jumping in trying to help with, “Try this, try that. Here is the USB stick, things like that.” And finally we are about 7 or 8 minutes late when I started my session which I apologize for that and I felt so bad for doing that. But I finally just put it on a USB stick, put it on a different laptop and did the presentation from PowerPoint on the other laptop. Not optimal but what I learned on that I guess is be prepared. Back to the Boy Scout motto, right?

Carlos: We’re going to the Boy Scout motto, there you go.

Steve: Be prepared, which I thought I was but be prepared with two laptops, which isn’t a financial thing because I had an extra laptop sitting at home. I could have prepared and brought up with me. But be prepared, bring two laptops and a backup plan, and a backup to your backup plan because whatever you think might go wrong, whatever could go wrong would go wrong eventually and it did in that session. I think I got a lot of feedback at the end that people were happy with it but I just felt bad getting started 7 or 8 minutes late because of that technical difficulty.

Carlos: And thanks to PASS for providing another laptop, right, PowerPoint loaded that you can move over pretty quickly. I think it worked well. The way that you gave that was kind of a story based idea, concepts. It was an introductory session, and so as you told the stories that kind of walkthrough some of those things. I thought your audience was super interactive. They were engaged, they were, “Urgghh”. When you tell a struggle they made it their own struggle or you could sense that they have been there, those kinds of things. And so I thought that it went really well.

Steve: Yeah, and what was different with that session was an introduction to database corruption. I hadn’t plan to do that sort of deep dive type bit level corruption fixing that I’ve done in the past. If I had planned that it wouldn’t have gone so well with the technical difficulties we had. But because it was more sort of the high level what is corruption and how do you deal with it. I think it worked out there really well and it was different than how I would have presented corruption in the past as well. I had a good time with it and I think people enjoyed it. At the end of course, though everyone comes up to ask questions to the speaker and you’re trying to get off the stage and all that because there was another speaker coming in we took the conversation outside to the outside of the room and continued for about 20 minutes with different people that had questions about corruption. It was really cool. I had a good time with that.

Carlos: I think that’s when we ran into Brent as well, Brent Unruh. Yeah, so good questions that’s always make you feel good as a speaker. People would come up afterwards and want to continue the conversation. It’s a mix bag, right? As an attendee you want to get that conversation in, then there’s other people in line and you want to go off to your next session as well. And then as a speaker you want to try to accommodate as many people as you can but we know that some of you probably slip out without having to have that conversation, so we apologize. I think another reason why we want to have an event like that is so that we can have that connection time in a less time constrained environment.

Steve: Right. And also I’d like to give a special thanks on that to Randolph West because last year at my corruption presentation and this year with the corruption presentation, at the end when you’re done speaking and everyone comes up to ask questions he helps filled some of those questions. And that was really awesome because he gave me a chance to get a drink of water and listen to what are some of the other questions were while I was packing up my laptop at the end. Thanks Randolph, I appreciate that.

Carlos: Yes, very cool. So Thursday night, a couple of shoutouts to the guys that helped organize the game night. Now ultimately it’s done through PASS but Kevin Hill was instrumental. I guess that’s how I found out about it. No, that’s not true. I found out about it through you, and then after investigating it Kevin Hill was kind of helping promote it and organize it a bit. And then he and Matt Cushing, one of those I have interacted with on Twitter. It was nice to meet him and shake hands with him.

Steve: Yeah, the game night was interesting because you come in and it’s just a bunch of tables and board games. And you first walk in the room and it’s a bit intimidating to look at all the people playing board games and wonder, “Ok, where I’m going to sit down?” And you just kind of wander just around a little bit, “Hey, there is a game that’s starting right now. I’ll jump in and play.” And it was awesome. I had a good time.

Carlos: Right. I think that worked really well. I think there was some confusion because people didn’t realize they had to pay to go. And I think that $12 was to buy a drink. Again, as somebody who doesn’t drink alcohol maybe they could have done without that, but I think it’s one of those things people liked it. But it did cause some confusion for folks.

Steve: Right, and I think also some of the $12 of that Game Night goes to the cost of the staff coming in and taking down regular seating and putting up tables because that was in the presentation rooms, so they had to come in and change out the room for that event.

Carlos: Right, and that makes sense, so the venue probably have cost associated with that as well. You’re exactly right. That makes a lot of sense. And then all of a sudden we’re at Friday. Well, the other thing I guess I do want to say about Thursday before we get to Friday is that I didn’t spend a ton of time in the Sponsor Area but Thursday was the day that I was able to do a bit of that. I guess special shoutout to the Windox guys, so Paul and his team. Got to meet with his partner and talk with him a little bit. It seemed a little bit smaller this year that in years prior, the Sponsor Area.

Steve: Yeah. That was interesting and I wonder why. I think the big players were there that you normally see and they had big booths. And then some of the smaller players that you see like Sean and Jen from MinionWare. I mean, they had their booth there. But it sort of the overall it just didn’t seem quite as large as what I remember in the past. Maybe it was just a different layout that made it appear that way too.

Carlos: Yeah, perhaps. On the other hot side is that we’ve also gone to some very large conferences like the Microsoft Partner Conference or Ignite. These are like 25,000 people conferences, and so all of a sudden you get that magnitude, right? And not to say that, I can’t remember what the attendance was but between 4,000 and 5,000. Not to say that that’s small but you’re talking about a quarter of a size and so it can pack things pretty quickly.

Steve: And then Friday, so we both had sessions. I had a session that was right around like 11:00 – 12:30 so right before lunch, and that was on Backup and Corruption. And although my laptop didn’t plugged in quite as easily as I would hoped because I spend hours working on it the night before to make sure it was going to work. It worked, it was just something to do with the video switching, Switching from full screen PowerPoint over to Management Studio. I had a little difficulty with there but it was better than the day before, and the topic was on Backup and Recovery. The thing that I had a lot of fun with was automatic restore script where when you write your backups, it would write the restore script at the same time. So that if you ever have to recover from backup you just find the restore script associated with the backup or transaction log file, open it and then uncomment a couple of lines and you’re running your restore. That was sort of the crowd pleaser I think in that presentation and it was a lot of fun.

Carlos: Oh very nice. Yes, so I was running a bit behind and I know that I had offered my laptop as a substitute if you needed it and by the time I got in there, you’re on the agenda, the presentation was fixed, and your corruption session was full. But the backup session was even fuller. I mean, there were lots of people standing and I thought, “Hmm. I’m not going to be able to get in this one.”

Steve: No, no. So the laptop problem was resolved and the presentation went smoothly from that perspective so that was a lot of fun. So then your session, Carlos.

Carlos: Yeah, so I was at the end and.

Steve: Establishing Baselines, right?

Carlos: Yup, through monitoring or managing performance through baselines was that kind of idea but taking baselines and thoughts around that idea, and so the last session of the day. I mean, obviously at that point and people are Summit is now wearing on them and everybody is a bit tired. So first I will say thanks if you were there, so again, to Will and Chris. I know who were there. Of course you were there, Andy was there, Mariano came, so thanks for you for coming out. That was my first time speaking at Summit and I feel like I do pretty well in group presentations. I’ve given lots of them and feel at home. I wasn’t so nervous and so I actually got up on the platform. And for whatever reason, that visual like being elevated. It was like, “Whoah.”

Steve: I’ll tell you. I personally think the last 5 minutes before a session starts, even if everything is ready to go and going smoothly, the last 5 minutes is the stress factor, right there. And then once you start talking. I mean I could see you getting nervous ahead of time, once you start talking then it really smooth over at that point and everything was cool I think.

Carlos: Right, so to help us some of that. Like I tried, which didn’t seem to go pretty well was creating a PowerPoint or some way to potentially engage the audience a little bit beforehand. Now I started just asking some questions so I had a little PowerPoint. I call it “Where in the world?” just like a photo and then the idea was you guess where it was. That didn’t seem to engage to many people which I thought was curious.

Steve: But there were a lot of interesting pictures that kept people occupied for that 5-10 minutes before the session started.

Carlos: Right, I guess there is that. But then once we get in there I felt a bit more comfortable. You know, you’ve given that session several times and that’s kind of when the habit start to kick in. I was happy. One thing I was happy about with the session is that we were able to still had some conversation. One of the things that I don’t particularly, I should say care for. My style I guess that’s a better word. I like my style to be a more interactive. I feel like if I’m just sitting there the only one talking in a presentation then I’m doing something wrong. I was glad that we were able to get some feedbacks, really some comments, right and kind of start a little bit of dialogue there.

Steve: Yeah, with that it almost seems like the larger the room is, the more difficult it is to get people to interact because they are afraid to speak out in the bigger room.

Carlos: Exactly right. That’s right and I guess, to that point we had. The room is, it wasn’t that humongous room because some of those rooms were big and I don’t remember exactly how many people would fit in there but we probably had, I don’t know, I’m going to say 50 people in there but I guess that was just on the cusp of still people being able to interact with each other. Once you get more than that it is like that Backup and Recovery session for example would have been a lot more difficult to do, I think.

Steve: It was a good session and I think it was the end of the conference session which is a tough timeslot but it went really well and I enjoyed it.

Carlos: Yeah, well thanks.

Steve: I thought one of the things I caught that’s really interesting is you’re talking about different products and tools and you bring up DBA Tools. I forget exactly what feature you mentioned but something with DBA Tools that you demoed and then Andy Levy, afterwards comes up or he said something about it because he works on it, and afterwards he came up and he said he actually worked on that thing, that feature that you were showing. That’s right so he had check in code. To him it was kind of a cool thing that the code that he has worked on has been shown at Summit. Only the best for companeros, right? So kind of cool to be able to show work and then that was it. We kind of went home. We went out to dinner with Derek and I headed to the airport.

Steve: Yes, so unfortunate story at dinner. We made a new rule that anyone who chokes and needs the Heimlich Maneuver has to buy dinner.

Carlos: Oh man, that was a bit scary.

Steve: We sat down for burgers and onion rings and I took the first bite of an onion ring. It wasn’t that big of a bite and somehow it got lunged in my throat. I wasn’t breathing. I did not have a working airway at that point.

Carlos: It was blocked. I was talking with Derek and you started kind of making a noise. I looked at you and your face was read and I was like, “Hey, are you ok?” I kind of pat you on the back and looked at your throat and I was like, “Oh man, we have a problem.” And we were in a booth which made it worse, we’re sitting next to each other and I was actually surprised that how quickly you were able to get out of that booth.

Steve: I wasn’t breathing. I had an extreme motivation at that point because I had an onion ring lunged in my throat.

Carlos: Yes, so that boy scout training coming back. That was a bit of a scary moment.

Steve: Yes, so very special thanks on that one, Carlos. I really appreciated that and I ended up buying dinner because of it.

Carlos: We’re glad that you’re still here and we hope that we never have to invoke that rule at dinner again.

Steve: Yes, I hope that it never happens again either, to me or anyone I’m dining with.

Carlos: Yes, that’s right.

Steve: But what a way to finish off a week.

Carlos: That’s right with a bang, that’s for sure. Ultimately that was our conference. We appreciate you companeros for sharing those of you who were there. For those of who weren’t, we’ll hope to connect with you at some other time.

Steve: Yes, and maybe next year at our SQL Trail Mix.

Carlos: That’s right, yeah. I think that’s going to do it for today’s episode. Our music for SQL Server in the News is by Mansardian used under Creative Comments. You can reach the shownotes for today’s episode at sqldatapartners.com/summit.

Steve: Or at sqldatapartners.com/117.

Carlos: And as always if you have thoughts about what you want us to be talking about on the podcast, you can reach out to us on various social media methods. One of them being LinkedIn. You can reach out to me I’m Carlos L Chacon.

Steve: You can get me on LinkedIn as Steve Stedman and we’ll see you on the SQL Trail.

Episode 116: Are people still using PSSDiag?

If you have ever worked with Microsoft support, you probably came across the PssDiag tool they use to collect information about your environment.  My experience with the tool is it always seemed, well–a bit clunky.  Now, it does collect lots of information and seemed to be a bit overwhelming when I was first a DBA.  Fast forward a few years and PssDiag was not a tool I use anymore and was not sure if anyone outside of Microsoft was using it.  I run into Jared Poche in Raleigh and he mentions he really likes the tool and mentions some of the improvements they have made over the years.  Well, you can guess where this is heading.

We talk with Jared about his experience with the tool and cover some of the basics about how you might go about using this tool in your environment.  There are two pieces to this tool, the first is the utility that collects the data and the second is the reporting piece SQL Nexus.

Are you using PssDiag?  Let us know if the comments below.

Episode Quotes

“It’s a really good tool in that it captures data from a lot of different sources and you tend not to miss much with PSSDIAG.”

“It is absolutely the go to tool for Microsoft.”

“Again using PSSDIAG and the Nexus will give you very visible data.”

“My biggest concern with PSSDIAG is people being too aggressive with the data they are trying to return from it.”

“Running it over the network is a really bad idea.”

Listen to Learn

02:47   What is PSSDIAG? What it runs and its usage?
04:54   Is Microsoft’s PSSDIAG and the publicly available different?
05:38   Using PSSDIAG in a small shop or in a large shop.
06:51   SQL Nexus
08:19   Improvements on PSSDIAG
09:42   How often Jared uses PSSDIAG?
10:13   Compelling reasons why to use PSSDIAG
16:04   Hand off point of getting information about the system, transitioning to smaller data set
20:52   PSSDIAG or DMVs? Which is a great place to start with baselining
24:12   Other common parameters in PSSDIAG, light capture vs. detailed capture
28:14   Tips and suggestions for starters
32:02   SQL Family questions

About Jared Poche

Jared Poché began working with SQL Server as an instructor for certification classes and has a passion for teaching and performance troubleshooting. Jared spent 10 years providing customer support at Microsoft, most recently as a Sr. Support Escalation Engineer. He is currently a database engineer working in Morrisville, NC and is leveraging his extensive knowledge to develop online coursework for SQL Server.

Transcription: Are people still using PSSDiag?

Carlos: This is Carlos Chacon.

Steve: I’m Steve Stedman.

Jared: I’m Jared Poche.

Carlos: Jared, welcome to the program.

Jared: Thanks for having me.

Carlos: Yes, it’s good to have you on and we’re kind of taking a break from our usual suspects at ChannelAdvisor. We met at the SQL Saturday in Raleigh and we’re talking. Ultimately our conversation is on PSSDIAG, but always good to talk with the ChannelAdvisor folks and your team is growing. You’ve added a few new community members that I know of and that’s always fun.

Jared: Yeah, Anders spoke at the SQL Saturday in Charlotte recently. I think we have 5 people there.

Carlos: Ok, very good.

Jared: I’m still kind of astonished by how active the DBEs and DBAs are in the community.

Carlos: That’s right, and so companeros, for those listening and I go back to a comment that Brian made, one of the DBA Managers over there. He said, “Some of the folks at ChannelAdvisor are very bright people and you don’t hear about them in the community.” And while Jared you’re getting out there a bit more of late, I kind of feel like you may fit that mold. Lots of the companeros may not know you unless they are kind of in the Raleigh or North Carolina area but hopefully that will change particularly with this interview.

Jared: Yeah, I’m trying to fit that mold. Yes.

Steve: Yeah, and I think the podcast is definitely changing that for some of the ChannelAdvisor DBAs that have been on.

Carlos: That’s right.

Steve: I think ChannelAdvisor has the record now for the company that has had the most guest.

Carlos: Unless we’re going to count Microsoft. Microsoft may technically still be ahead with all the PM they’ve had on.

Steve: Oh sure, yeah.

Carlos: Other than Microsoft… those two different layers are way ahead of everybody else. So ultimately one of the conversations we had, again kind of going back to that Raleigh conversation, and we were talking about performance and collecting information about the database. I can’t remember exactly what your comment was but you mentioned PSSDIAG. And I think my comment was, “Ugh, does anybody still use that?” And you said, “Yes, it’s great.” And I thought, “Oh, we’ve got to have you on the program to talk a little bit about it.” And so for those who haven’t use that, I guess let’s do a little groundwork here. Let’s just talk about PSSDIAG, what it is, and maybe even a little bit of history of it.

Jared: Ok, so PSSDIAG is this fairly own composing tool that Microsoft has developed and makes publicly available. It’s up on, I can’t think about the site is currently, but it’s publicly available for download. And this own composing tool that is used primarily for SQL Server performance issues. There are a few other odd circumstances you might want to run this but mainly you use it for, and Microsoft encourages to use it to gather data about performance issues they are having with SQL Server. And it runs a performance monitor output in the background. It runs a profile or trace on the latest version an xEvents trace if you want. It runs a bunch of queries to get information in DMVs. It will produce like 50 or 80 files of output in the folder that produces and it can produce quite a large volume of data. And the idea is you run this thing, you might run it for half an hour or whatever or if you had a different window when the problem occurs. And you know the problem might happen sometime in the next hour or so, you want to run it and capture data while whatever problem is going on. It’s a really good tool in that it captures data from a lot of different sources and you tend not to miss much with PSSDIAG. There are a few odd things occasionally that you might want to add on to say the normal configuration but it’s a really own composing tool and it puts a lot of data at your disposal.

Carlos: Right. Now, you are a former Microsoft employee, right?

Jared: I am. I was working at the CSS site in Charlotte for about 10 years. And a little bit of a contractor as well, a year and a half.

Carlos: I have heard, I guess maybe you can clarify for us. So traditionally you’re going to be exposed to PSSDIAG for the first time, at least I was. If you’re company has Microsoft support you call them up, you know, you’re looking at a performance issue. This is the go to tool, right, for them?

Jared: Yes, it is absolutely the go to tool for Microsoft.

Carlos: One of the things that I had heard was that if you go to their support site and what’s publicly available is slightly different, true or false?

Jared: They have an internal version of the tool, yes.

Carlos: Ok, but from the nuts and bolts perspective ultimately the same?

Jared: Ultimately the same. There might be some slight differences in some of the specifics of what options you have available. GitHub was what I was thinking of earlier. The current publicly available version is available on GitHub, and in the past there were some slight differences in kind of the custom configuration options you had in the internal tool and the publicly facing tool but they are 98% the same.

Carlos: Got you. Ok.

Steve: So then, I mean if you’re doing a support ticket with Microsoft they are going to ask you to run this. But if you are a DBA out there in a small shop and you’re having some performance issues, is this something that’s going to be easy to use and just that they could run and dig through the results and get some value out of it or is it more just for Microsoft to use?

Jared: It is something that you can use. As far as somebody who is in a smaller shop, I would actually have less concern about somebody in a smaller shop using it than somebody in a bigger shop with very big, very active, very busy servers. They don’t know what’s configured. The question that is being asked many times, “Will PSSDIAG affect my performance?” And the answer is possibly, and it depends on how you configure it and how busy your servers are. So in a smaller shop I’d be less worried about it. When I started using PSSDIAG in 2004 I think, I wasn’t very versed in performance troubleshooting at that time either and so I spend a lot of time looking at the perfmon manually. I’m looking for the profiler manually. There is also basically a sister tool with this which is SQL Nexus which is also available on GitHub. And what SQL Nexus does is basically grind through a lot of the data that PSSDIAG provides. Does some analytics on it, you know, figures out what’s the biggest resource consuming queries are. Your big wait types and puts it into a nice GUI set that you can see that stuff and consume it much more easily. You can select through it manually but SQL Nexus does make a good job of making that easier to understand.

Carlos: And worth noting in the beginning, I can remember being in that same state as far as just kind of getting my feet wet so to speak from all the tools that are available and taking a peek at that. One of the things that is a bit I’ll used the word overwhelming, or can be overwhelming of PSSDIAG is because it’s collecting all that data. It’s almost like where do you start.

Jared: It is difficult to know where to start.

Carlos: Yeah, and you kind of have to. I think if you have a good understanding of what all of those pieces are. It makes a little bit easier but starting there like your first troubleshooting starting place. Yeah, it’s going to take some time to get through, so be patient.

Jared: It can be intimidating at first. I do think that SQL Nexus makes a lot of that easier by kind of figuring out what’s important for you in doing a lot of the analysis by itself.

Carlos: Well now, let’s talk about some of the improvements. Again, so my experience with PSSDIAG is probably, gosh. You know, 7-8 years ago at least now. I remember SQL Nexus being a bit, I hate this word, clunky. But I felt like it was beyond maybe what I could do at that time. I felt like I didn’t have the experience to get it all set up. I never quite seem to fit that right. Is that getting better? Is that easier to use now?

Jared: For me it has gotten a lot better over the years. The last couple of releases have done more, there is more things exposed such as they added in something to get out the details of say sp_configure which is something that PSSDIAG has gathered for a long time but it will flag things in that that are unusual and things you might want to reconsider. I think that the GUI interface is better than it was. I think there is a lot to positive say for that tool, so I feel really good about SQL Nexus. And I kind of feel like I was doing myself a disservice because when it came out at first I was just kind of like, “Ahh, I don’t need that thing.” Because at that point I’ve been using PSSDIAG for a couple of years and I knew what to go and get and what to look for although it does in the long run it did saved me a lot of time using Nexus.

Carlos: I see, interesting.

Steve: How often do you end up using PSSDIAG in your job?
Jared: Currently I do not use it very often. I mean currently if I’m looking at a performance issue, and this is something that I started doing a lot when I was still working at Microsoft. A lot of times I’ll just use DMV queries and figure out what I can from that about an ongoing situation, and I feel like that gives me information very very quickly I can use to diagnose problems.

Steve: Ok, great.

Carlos: Now, again going back to our conversation. One of the things you got to mention was one of the things you thought PSSDIAG was really really good at. So we’ve talked about kind of collecting the metrics and whatnot, so what are some of the compelling reasons of why you use PSSDIAG? Other than just collecting a whole bunch of metrics, like you everything that you might need.

Jared: Well, it is all encompassing and doesn’t miss a lot so that’s a positive. The perfmon collector gets you everything you’re going to need in the vast majority of cases. The fact that it gets the information from the windows event viewer can really be helpful on occasion too especially if you’ve got a situation that you have disk issues. You might have errors that will come up and say the system log. That’s not something you’d necessarily think of unless you had a tool that’s helping to expose that information for you. One of the things that I got to really be fond of. In the latest version they have improved the ability get xEvent traces, so that’s something that’s been in the last couple of releases. But the last one that I saw before the current release, it was till kind of a stub, like there was a tab for configuring profiler and you really want to get away from profiler. I wouldn’t even think about running PSSDIAG with profiler on the servers on ChannelAdvisor in production. There’s no way.

Carlos: Interesting. I guess I want to come back to that, right.

Jared: Well, that’s a matter of overhead. I mean using xEvent traces and I’m more familiar with looking at profiler traces but I’m getting pretty familiar with xEvent traces. And the xEvent has a lot less overhead and the overhead of profiler can affect the performance of our system. That’s the thing in the past. And the busier your server is, the more likely that is to be a problem. So the fact that they’ve improve it because the xEvent thing used to just be a stub. And actually I remember one of the releases a couple of years ago there was a tab for configuring an xEvent trace and if you try to use it it would say, “Wait, this isn’t actually working yet.” So now it’s fully functional, now it’s fully complementary or equivalent to what you could get out of profiler so that’s good. One of the things I really really like out of it is in the output file there’s, it names a lot of things based on the name and the system and then the version of SQL Server, so the file is really long. There’s one file that is the perf_stats_startup.out, and what the script does it has like three different sets of DMV queries that runs. And it does one every minute that will do things like getting information from DMOS wait stats and it has another one that run maybe every 30 seconds that will get information about memory consumption. But the main loop, the thing that it runs like every 5-10 seconds runs queries against dm_exec_requests to tell you what all the active queries are especially the ones that are waiting or causing waits in the system. It does a second query, that for all the things that it got with the first query. The second query will return the statement that’s being run and the procedure it’s a part of which is very helpful for narrowing down what you want to look at and finding the section of code you actually care about. And the third query that tells you what the head blocker is if any blocking chains you might have. One of the things I would do at times to diagnos the situation very quickly is I would just open up that file, look at the information we’re getting from dm_exec_requests about active queries. I would have that open in Notepad++ or Textpad and I would just scan through that very quickly. Because it will run these queries like, I don’t know it it’s every 5 or 10 seconds. But each one you’re just getting one snapshot of what you’re running queries and what your waits look like. But if you see one snapshot, and another and another and you can thong through them all really quickly, it’s easy to come up with an overall impression of what’s going on. And to say, “Oh well, yeah I’ve got some I/O waits here and there but that’s not a big of a deal. But I’ve got cxpacket waits lined up for days. I need to figure out what’s going on with that.”

Carlos: That’s interesting as you mentioned that and I agree that that idea of the snapshot is very important. One of the things that I actually will have done at the PASS Summit is talking about baselines. So how do you, I guess thoughts around leveraging a tool like this which is kind of a snapshot but in a sense to give you, you want a broader than just what’s my current CPU, right? You want a little bit more history than that but it’s not something that you’re going to be running for weeks at a time either.

Jared: Yes. I would really consider using Nexus with that because of the fact that you can gather PSSDIAG whatever time you want to. If you want to gather an hour or two worth of data at whatever time of the day you feel will be much appropriate, go ahead and do that. But then use Nexus with that and you’ve got a very visible, you’ve taken that data set and made it very visible and very easy to look at and to refer back to it later because it’s like you’ve got all the perfmon data is in there, and average out. And you can say, “Ok, for these two hours this is my CPU consumption. This is what disks look like. This is how much free memory I had. Here’s what my main queries were, here’s what my mian wait types were, here’s what my resource consumption per procedure look like and you can have that kind of information for multiple sources and have it easily accessible through SQL Nexus.

Carlos: Ok, so you mentioned you kind of switch off of it, so when do you feel like it’s the hand off point? It seems like it’s a very handy tool particularly when you’re, I don’t want to say going in blind, but again if you think about the origin, right? So I am a support person, somebody calls me up, I don’t know who they are, I’ve never seen the system before, I want to get as much information as possible. Where do you see the hand off or the evolution perhaps of I need to get information about the system even if whether I’m new or whatever. I’ve just been tasked to this database to using PSSDIAG to, “Ok, now I know something about it. Maybe I don’t need to use all the pieces. I am more comfortable with the DMVs or something like that.”

Jared: Well, I think the more that you know the environment that you’re dealing with; the easier it is to transition to using a smaller data set. I think part of the reason PSSDIAG is as usual it is for say a Microsoft engineer is they don’t know what you environment looks like, and they don’t necessary know what they’re looking for. They don’t necessarily know what say unusual sp_configure settings you had. There is a lot of things that could potentially be the problem and there are more things that they might need you to consider. Now, in my case, part of the reason I made the transition, started doing more of my troubleshooting with DMVs at least were possible and to me there are still circumstances you just can’t replace PSSDIAG because of some of the things that it does. But part of the reason I made the transition was just for efficiency. So let me give you a hypothetical. Let’s say that you’re the customer and I’m working as Microsoft. We gather half an hour of data off of your very busy production server. You end up with say, three or four gigabytes worth of data in your output folder. You start zipping that up, you’re going to have to upload that to Microsoft. There’s a tool within Microsoft that when you upload that it will go ahead and unzip it and run it through the SQL Nexus output and I’ll eventually as the support engineer would get an email with basically the Nexus stuff already run. And the Nexus analysis that could take some time, a large data set, that can take a couple of hours. So I was actually on a critsit, if you’re familiar with the term. I was on a critsit with somebody in one case and they had a high CPU issue. He had been troubleshoot it himself and actually the problem went away before he contacted Microsoft and I was still working with Microsoft at that time. But the time I get on the phone with him the situation has been over for two hours. Now, PSSDIAG, I could configure PSSDIAG and send it to him and say, “Ok, if your problem happens again start running this and give me a call.” But I kind of wanted to see if I can do something more than that. I was thinking and I said, “So how did the problem go away? What happened?” And he told me, “Well, it stopped on its own. I was trying to figure out what was going on and it just stopped. I didn’t do anything.” So I asked the guy, “Ok, did you restart SQL or did you say flush the procedure cache?” Because that’s something a lot of people will do and it was kind of frustrating when you’re a Microsoft engineer. “Is the problem still going on?” “Oh no, I rebooted the server.” “Oh well, I guess I can’t gather any actual data.” But it hadn’t restarted, it hadn’t flush the procedure cache and so I thought to myself, “Well, whatever plans were consuming all the resources, consuming all the CPU, they should still be cached.” Let’s go run a query against dm_exec_query_stats that I had and I ran that and we looked at the output. And he had about 5-6 queries that were consuming significantly more CPU than anything else. Like you look at them in descending order and after about query 5 the CPU dropped by three digits. That’s actually unusual very often when I see a high CPU case and I’m looking on the DMVs. You’ll see that there is one or two things that’s really killing it. But in this case, I used the DMVs, I’m able to see the cache plans. We can look at the cache plans. We can go find the procedure that is related to it. We look at the table, we look at the indexes. I was able to get to everything I needed just from running a query basically against dm_exec_query_stats. And I didn’t have to have him gather data and upload it to me and I spent hours waiting for the data and then analyzing the data and then get back to him. I had an idea very quickly of what was going on. And after that one, that made a really compelling argument to me about the efficiency of using the DMVs.

Steve: Ok, so then just sort of comparing that efficiency and then getting all the information there. I mean, if you’ve got a new server that you are now responsible for you’re going to take over maintenance on. Would you say that, I mean, PSSDIAG would be a great place to start to get that sort of baseline and so that you know what’s going on with it and then later deal with the DMVs or you just sort of skipping it sometimes and go directly to the DMVs.

Jared: I think you could do either approach? Again using PSSDIAG and the Nexus will give you very visible data. I think the way it presents things might be easier for some people to see and understand and realize what normal and how this is different from that. But with the DMVs I think one thing that would be effective would be to have ongoing normal processes where you collect information from say dm_exec_query_stats or dm_exec_request and put that into a permanent table that you’re willing to keep for x amount of days. We some of this at ChannelAdvisor to keep track of say dm_exec_request information so that we can see waits that are going on inside the system and we can look back at a given period of time and see what kind of waits we normally see on a Wednesday at 2 o’clock in the afternoon. We can see that. You could do either approach. Like I said I like Nexus for the fact that it has a decent GUI on it. But you could also just take the DMVs have automated things to gather that sort of information. It depends on how long you want to retain that information. You have to be a little careful about trimming your tables if you’re taking that raw DMV information and storing it somewhere as opposed to say running the PSSDIAG and running the Nexus. Now you have the nexus output, you can actually get rid of the PSSDIAG data, however any gigabytes that is. You can also, Nexus takes the data and puts it into a database and it actually needs a database within SQL server that can connect to and juggle the numbers around. But you can backup that database and then at whatever point you can move it to another server. You can point SQL Nexus at it and it’s very effective in that fashion. Ok so for baselining to me you could use either one. It just depends on your preferences.

Carlos: So we talked about wanted to hit some of the new features and I think you’ve mentioned a couple of them, obviously Nexus being one. So as far as like, I’m sure even the documentation has improved a bit so you mentioned some of the parameters or settings you could have. So if I just run PSSDIAG with no parameters I’m going to get the event log, I’m going to get the SQL Server log, I’m going to get the profiler trace. I am assuming it’s still the default. You mentioned I could get xEvents, now I’m assuming parameter change and then I’m going to get the performance monitor metrics. So what other common parameters am I using when I am talking about PSSDIAG?

Jared: Well, the thing currently is and this is where people go wrong with it or can go wrong with it is now there is basically default packages or default scenarios or scenarios that you can just basically say I want to gather general performance or I want to gather detailed performance. And one of the things that I would suggest people not to do with PSSDIAG especially on the production environment would be to go in and start by telling it to do a detailed performance output before you’re familiar with the tool and have an idea of how much data it’s going to capture in your environment. I would go with a general or light option. The big difference between a general or a light capture and doing a detailed capture is that when you do a light capture you will gather the events for whether you’re on profiler or xEvents, you will get RPC started and completed and you will get SQL Batch started and completed. And you will get errors and you will get a bunch of other things as well but the main things will be interested is batches and stored procedures started and completed. When you go to detailed, you’re going to gather a lot more events because you’re going to gather statement level events. You’re going to gather every statement starting and completed within SQL Server and you’re going to gather execution plans of whatever type you’ve selected or whatever type is the default for the detailed package currently. And think about it, if you got a store procedure and that store procedure has 50 statements in it then by going to a detailed trace you’re going to gather, instead of gathering two events for that stored procedure – starting and stopping, you’re going to gather those two plus 50 for each statement starting, 50 more for each statement stopping and for each non-trivial statement that you’re going to capture the execution plan. So there is a big difference in terms of how much data that you’re going to be gathering and thus it’s going to be more likely to affect the performance of the system in some way, and it can. And if you’re SQL Server is pushing 2,000 transactions per second that’s probably not a problem. If it’s pushing 25,000 transactions per second the same package might bring that server to its knees. And especially if I’m not familiar with running PSSDIAG in my environment I would start off by doing a general or a doing a light and seeing how that works before I try to crank it up.

Steve: So I guess from that perspective it also makes sense if someone is just learning it if they have access to a test server that might have far less load to just give it a test there first to make sure you understand what do light versus the more detailed is going to do.

Jared: Absolutely. And there are some interesting circumstances like there are things in there that are made for gathering information about always on or gathering information specifically about memory issues or things like that. I think there is used to be one for, like there is still a collector for service broker and there are circumstances you might want to gather those things because it provides a lot of information. There is one for link servers that will run all the queries to get information about what link servers you have set up. So there are a number of useful little things like that. But my biggest concern with PSSDIAG is people being too aggressive with the data they are trying to return from it because it can affect performance. I had cases when I was working at Microsoft that I would go in with the PSSDIAG package and we’d run it and we gather data and I analyze the data and realized, “You know what I can’t use this because all I can see is the SQL trace waits.” I can’t tell what else is causing problems on the system because the tool is gathering too much information that were waiting for that information to be written out. And I can’t see any other bottlenecks because of this bottleneck.

Carlos: Ok, so I guess then at this point for someone who is about to go and give it a shot maybe after listening to the podcast. I mean, besides keeping the sampling light there. Any other tips or suggestions to help someone get started?

Jared: One or two that’s big. Another thing that affects performance is running this over the network as oppose to running it locally on the server. Running it over the network is a really bad idea. I would dissuade you from doing this if it all possible. The idea is that SQL has these delays when it’s trying to write the information out. Now, PSSDIAG does server side trace but if you’re going to be using profiler or using some other tool to do your tracing, I would really suggest not doing it remotely. You can run PSSDIAG remotely and tell it to connect to this server and this name but it introduces a much longer latency in terms of how long it takes for the trace to write out that data because now you’re… And that is affecting how the SQL Server engine operates now, so that’s a whole different magnitude. The other thing that I’ve seen that causes problems and it’s not specifically with PSSDIAG is not to run a whole lot of traces at one time whether you’re using xEvents or profiler. There used to be commercial tools. I won’t name any, and I probably couldn’t remember them all of them. But that would run four or five or more profiler traces at a time and that would affect the performance of the system because SQL Server is trying to write out things and keep track of things for each one of those different trace files. Especially when you’re doing multiple and you’re doing it over the network, that can really be consequential.

Steve: Yup, interesting. Well, ok so a side note put on that running profiler traces. I mean, what would your take be if someone wanted to run a profiler trace like 24/7/365, all day, all year just to keep track of every query that’s ever being run? How big of an impact something like that can have on performance?

Jared: Again it would depend on how busy the server is. On a busy production environment I would ask them to find exactly what it is they are trying to find out here because I don’t think that’s what they are trying to find out. I think that’s a lot more information than you could reasonably use. How would you aggregate that information? What are planning to do with it? And in any case I would stay away from using profiler at this point in any case because xEvents is now a super set. It has been since SQL 2012, has been a super set of what profiler can do. We have more kinds of events of xEvents now. It provides more information to you and it’s more efficient. It was designed from the ground up to be faster and to have less overhead than profiler. I really encourage you. If you’re more familiar with profiler, I entirely understand. But I would really go ahead and pay that price because I think it’s more than worth it.

Carlos: Well now I’m interested to find out what happens with profiler. I don’t think it’s going away and the reason I say that is that they just announced. Or maybe not just but one of the things we talked about SQL Server in the News is they’ve added query store as a source for profiler.

Steve: Alright, shall we move in to the SQL Family section?

Jared: Sure, go for it.

Carlos: So Jared, how did you first get started with SQL Server?

Jared: I got started when I was a part-time community college instructor. I had been teaching for one semester and my older sister actually work at one of the colleges I was working at and she asked me if I knew anything about databases and my answer was no. At that point the college and this was Forsyth Tech in Winston, Salem. They did some certification classes for SQL Server and one of them has actually in one of their 2-year degree. And they needed somebody to teach this classes for the next semester, so she came to me and asked if I could do that and I said, “Ok, I’ll look into it.” And I basically picked up a book and taught myself enough of SQL Server to get certified in it and then I taught the classes there for about three years. And from there at that point, the interesting thing about teaching is whenever you’re trying to teach people something you have to know it to a greater degree than your teaching. And also they’ll ask you questions about the subject that you’ll never think to ask yourself. So it’s like you as a teacher end up learning even more from having students because you’ve already figured out all the questions you had in your head but now you’ll get a whole bunch more questions and you have to figure out how to answer them too. So I taught that for about three years and that was when I applied for this helpdesk e-job that I saw on Monster which actually turned out to be a contract position to go work at Microsoft at Charlotte.

Carlos: Interesting. So it’s very similar almost to like getting started with presenting in a sense, right?

Jared: Yeah.

Steve: So if you could change one thing about SQL Server, what would it be and why?

Jared: That’s actually pretty difficult. Trying to think of, you know, I’ve had pain points here and there over the time. I’ve seen people do some horrible things with service broker. I’m trying to think, what else? At the moment I guess the biggest thing I would come up with is page latch waits because there is such a big consideration and a kind of a thorn in the side currently at work so I guess I would try to some find way to eliminate that level of pessimistic concurrency. We’re trying to move to doing a lot of things doing in-memory optimized tables and whatnot to move to more optimistic concurrency because with our throughput, with our load, we need to have that. We need to get rid of the contention we get from page latch waits. That’s a big consideration.

Carlos: Got you.

Steve: Ok, very nice.

Carlos: What’s the best piece of career advice you’ve ever received?

Jared: Difficult to say, I think one of the things I would go back to is when I was still contracting, when I was still starting off at Microsoft. I was speaking with one, my tech lead at that time was Bill Carroll. He is now an Escalation Engineer. He’s been at Microsoft forever. I was working on some performance case and at that point hadn’t a lot of experience working. I had no experience before I came to Microsoft working on performance cases. I was finding them difficult and Bill Carroll said, “We will always have room for somebody who knows how to handle performance cases around here.” And so that I think really probably was the best piece of career advice because at that point it gave me a niche. It gave me something to figure out and try to focus on and I’ve done a lot with that so I’ll give Bill Carroll the nod there.

Carlos: There you go.

Steve: Alright, and our final question. If you could have one superhero power what would it be and why would you want it?

Jared: That’s funny to me in part because I’m an avid video gamer and my wife and I used to play City of Heroes so we have played superheroes many times. For selfish reasons I would like to say either flight or teleportation because that would really be cool. But honestly in real life if I could have a super power that would be the power to heal other people just because who I am as a person if I could do something that could make life better for somebody else. If I could take away something horrible that’s happened to them that would be a great thing to do.

Carlos: Very cool. Well, awesome.

Steve: Alright.

Carlos: Well Jared, thanks so much for being with us today. We appreciate the conversation.

Jared: Well, thanks for having me.

Steve: Yeah, that was great to be able to chat with you, Jared.

Jared: Same here.

Episode 115: The Companero Conference Retrospective

Companero ConferenceIt was a beautiful idea.  Gather together with a group of our podcast listeners in a conference type format and find ways for us to connect and help each grow.  I had seen it down in other events; however, I wanted to add my own flavor.  I wanted this to be different.  While the event turned out great and we got good marks from the attendees, we didn’t have the response we were hoping for.  In this episode of the podcast, we have our companero conference retrospective.

We try to give our honest feedback about what we did well, what could have been improved, and what the attendees reported.  I can’t say enough about the help I got from our speakers–Jonathan Stewart, Kevin Feasel, Randolph West, Tracy Boggiano, and Doug Purnell.  They were awesome and the event was much simpler because of them and their commitment.  They are truly my companeros on the SQL trail.


We would like to send you a survey on what you think the future of the conference should be and whether we should keep it around.  We look forward to hearing from you.

Episode Quotes

“I wanted a way that people could come together, could talk with each other, connect, build those relationships.”

“We felt like we were able to create that intimate environment where people could participate.”

“You’re not just a face in the crowd. You’re actually there participating and I think that’s the difference.”

“I think the pull of those bigger conferences was just too big and lot of people ended up going to either one of them.”

“I don’t really like the word conference… because we’re going for the unconference idea.”

“I think we learned a lot. I think I’m glad that we went through it and thanks obviously to those who attended and to the speakers.”

Listen to Learn

00:39   Companero Shoutouts
07:25   SQL Server in the News: Import Flat File Wizard
09:06   SQL Server in the News: Service Packs to Cumulative Updates (Pros and Cons)
13:45   Links of the show notes
13:57   The reason/objective of putting up the Companero Conference
17:59   Challenges that happened
20:57   Conference outcomes – Are the conference objectives met?
25:04   What are the big challenges?
36:23   Thoughts or changes for future events, to do it again or not in 2018?
45:30   What to look forward to? What’s next?

Import Flat File Wizard

Transcription: Companero Conference Retrospective

Carlos: Companeros, welcome to episode 115. It’s good to have you on the SQL trail today.

Steve: Yes it is and it’s good to see you Carlos.

Carlos: Yes, and it will be actually good to see you in a couple of weeks here we go on to Summit. We don’t get to see each other as often as we would like.

Steve: Is this where we let out that we’re not actually in a recording studio.

Carlos: Yeah, oh boy! Was that a secret that we’re keeping there?

Steve: Yes, yes.

Carlos: Yes, in the MGM Studios or on Park Avenue or whatever it is, or the MBC Studios.

Steve: Yeah, so we’ll actually be seeing each other in a couple weeks as well as many of our companeros who maybe attending PASS Summit as well.

Carlos: Yes, that’s right, and we do have an event that will make a little bit more sense which we will talk about at the end of this episode. So today, we actually want to do a Companero Conference retrospective. We’ve been kind of evolving this idea of the conference right in front of our podcast audience and we went ahead and had the conference this October, and so we want to talk a little bit about it. How it went? Maybe some of the lessons we learned from it, the challenges that we had and then what if in the future it might have.

Steve: Yes, but before we do that. Do we have any companero shoutouts this week?

Carlos: We do. I want to give a shoutout to Kira, and you’ll forgive me but I don’t have her last name but the organizer of the girlsanddata.org site. She had put this together. I thought it was interesting in a way because there’s like girls who code at code.org, lots of information about coding and teaching kids how to code. And she felt there were lots of opportunities in technology particularly in data that you don’t need to be a coder, you don’t need coding skills to do. So she connected with Mindy Curnutt of Dallas. And Mindy has taken the ball running with this and they were kind of just in the Milwaukee area, kind of doing things locally, and Mindy has connected with them and they’ve done events now in Dallas and in Charlotte this last weekend. And I went down there for the SQL Saturday actually for the purpose of putting two of my girls attend the event. They had a great time, I thought it was phenomenal what they did. You know, my girls are not, they are kind of that age where they are resisting anything that dad tries to get them to do. So there’s a bit of that, right, going on they have that to overcome. And so my oldest daughter, I should say Andy Leonard’s daughter, if you want to hear really great things you should go hear Andy Leonard’s daughter. She was absolutely over the moon with it. My daughter is responsible as well it didn’t blow me to death. And so which may not sound very good but some of the languages she’s been using lately, that was a pretty much as good as it gets.

Steve: Ok, and what age is she at?

Carlos: She’s actually 15. The girls at data.org is actually directed more middle school girls, so the 6th to 8th Grade girls. She is a bit older than that but I thought, “You know what, I’m just going to put her in there,” regardless of what I think it will be. It wouldn’t be that bad. May other daughter is 13. She is an 8th grader currently, and again, I think she enjoyed it. But some of that feedback was hard to get because she couldn’t say. She wasn’t super negative about it but I think she couldn’t be super positive about it because for fear that I would sign her up for something else.

Steve: Interesting. You know what I think is really cool about that program is that, I mean there are so many different programs out there to try and introduce younger people to programming. Like you said, data can be very different than programming. I think that in this school system as well as like community college and university even further up on that level the education on data is usually pretty weak compared to the education in programming.

Carlos: Right, exactly. And so I think it was an interesting that she put together, you know, focused on that. They are using tableau as their reporting source and they have several pieces of information they put together and do different things there. And so I thought it was very interesting. They had an interesting mix of people, so in that class they were 15 or 17 girls, probably nearly half of which parents were attending SQL Saturday. They had actually made the announcement to several of the middle schools and middle girls camp so other folks weren’t currently tied to SQL Server were still attending that. So I thought that was good.

Steve: Alright. Can we just pause for a second I need to turn off some background noise.


Alright, I’m back. Sorry about that. Hey Carlos, I’m back, sorry about that.

Carlos: Not a problem.

Steve: And out next companero shoutout is from Eduardo Cervantes.

Carlos: Yes, Eduardo on the West Coast reaching out to us and reminding us that some of our listeners are developers. He mentioned, “I do most of development. I appreciated the podcast”, and connecting him with new features and things. So thanks Eduardo for reaching out and for also just responding to something on messenger. I’m sure none of our companeros are doing it but I have people reach out to me from time to time they want to connect on LinkedIn. I’m making a personal habit to respond. It’s pretty generic but you know it’s, “Hey, thanks for connecting. I hope you’re well. Let’s chat, like what do you need help with?” It kind of bothers me to no end when I just don’t receive a reply. Even if it’s just a, “Hey thanks” or something, right? Let me know that you’re there, that you have a pulse. So with that I guess a little SQL Server in the News is on file.

Steve: Yeah, so what’s up with this import flat file wizard you mentioned?

Carlos: Yes, so one of the new features they’ve put in and let’s really just. I don’t want to say dumbing down but they have the data import option so kind of using it in the background with SSIS like package that will get import data. So what they’ve done is they kind of slim that down a little bit and made that specific to flat file data so CSV and text file data. What it helps people do is to walkthrough the wizard because before on the import data you had to go in and kind of like, I can’t remember exactly what the button was, but to see the conversion with the columns are going to be. You don’t have to drill down a bit anymore. It will do those conversions. So first you got a preview of the data actually, so again very similar to features that are in SSIS. It’s going to import that data. You’re going to get to look at it, make sure the columns are all lined up. The next screen is, hey I had to change the column types based on what the wizard is telling you it thinks it should be and then you can import. Yeah, I think it’s just a little bit easier and make it a little bit faster. Do those little one offs since like we’re always importing just random Excel data for whatever reason, look up data and things like that.

Steve: Oh yeah, and there are so many different ways of doing it I think. And that’s just one of them that makes it easier so that’s great. Another SQL Server in the News items that we have is around service packs.

Carlos: It got very loud on your side or something.

Steve: Hold on a second, Carlos.

Carlos: Sure.

Steve: Alright, I’m back. Sorry about that.

So Julien let’s cut that out we’ll just go back when we start talking about service packs. I’ll explain afterwards what’s happening, Carlos.

Alright, so another SQL Server in the News item we have is on service packs. This came out a couple of weeks ago with SQL Server 2017. Around that time they also announced they are going to change the way that service packs work. Instead of doing service packs they are just going to go straight to cumulative updates. And for the first year, it sounds like the plan is on SQL Server going forward they’ll have… They are not going to have a service pack, they are actually going to have a cumulative update every month. So you get 12 cumulative updates and about at the end of the year those CU12 should be like the equivalent of what would normally be SP1 in the past. I think that there are some interesting takes around that because one it sounds like we’re going to, and this actually happened for that last couple of years now, is that cumulative updates have been as robust as service packs. Didn’t used to be that way years ago. But I think it gets away from service packs having sort of people to wait, I’ll wait for SP1 or I’ll wait for SP2 before I upgraded that version. Now there will be no SP so maybe people will say, “I’ll wait till CU1 or CU2” but who knows.

Carlos: Yeah, we’ll see interesting I think. I feel like this is kind of the dev ops coming, I mean not to the database perspective but that dev ops mentality making its way into applications. Obviously it’s kind of everywhere people are talking about it and now it’s just making itself involve or connecting with what we do on a regular basis. And so more applications like this will kind of go this route.

Steve: And I think it really highlights the importance of the software insurance.

Carlos: Oh yeah. We talked about that before we started recording and I think with this rapid updates, again, people are already talking about SQL Server 2018. It is interesting to see what will happen. I feel like, again, kind of with the Azure mentality of everything being in the cloud then releasing software so quickly that you’re going to almost have to have software insurance just because have you know when to upgrade anymore, you know just pick a year. List some feature that you’re really really interested in kind of those rolling updates. Yeah, it will be interesting to see what happens, what enterprises do with. So let’s just say in 5 years, and I’m making some assumptions here, but let’s just say there is a release of SQL Server every year through 2020. What do you there? It used to be like upgrade so like a big deal like you plan time every couple of years. Now I think we’re going to see a lot more versions out there than we ever have before.

Steve: Right, yes, which I think has some pros and has some cons. I think for the people who stay up to date they’re going to have more work to keep up to date there. And then I think for the people who don’t keep up to date, well you’re going to get outdated I think much quicker.

Carlos: Much quickly, yeah that’s right. So the gap is going to get a bit harder. But to their defense, Microsoft has been very good about the backwards compatibility and I think that’s even getting better.

Steve: Oh yeah, and I think hopefully with the more frequent versions it will make it easier for people to jump into the next version because you don’t have 5 years of changes or even 2 years of changes that you have to worry about compatibility with you’re dealing with once a year.

Carlos: Exactly. Yup.

Steve: Alright.

Carlos: So some interesting stuff. Ok, so our show notes for today’s episode will be at sqldatapartners.com/conference.

Steve: Or at sqldatapartners.com/115 for the episode number.

Carlos: So ultimately what we wanted to do today is talk a little bit about, and I guess have that retrospective for the conference that we put together. I feel like we should start with the premise of why we wanted to put this together. Admittedly, I was heavily influenced by my own experience, so getting involved with SQL Saturday, with the User Group, going to SQL Cruise, going to Summit. So 2013 was the first year I had gone to the Summit. I just had different experience then with other conferences outside of the community events. I felt like there was a way to continue that idea of having people participate more in the conference. This kind of goes back to Allen White, so this is all Allen White’s fault. With that idea that we all have something that we can share with each other and those traditional conferences, in my mind they do a great job of expelling information but as from a collaboration perspective and from how this apply to my own unique perspective, they don’t do a great job. And so I wanted a way that people could come together, could talk with each other, connect, build those relationships they could take home with them, things like that.

Steve: Yup, and that’s the key, the connecting and the relationships. I mean we stressed that as one of the most important factors in the conference from day one.

Carlos: Right. What it turns out is a little bit of a tough sell to a wider audience. I think as technology people they, I mean like they behind the screen and just consuming information. And so I think it takes a little bit of a challenge. It’s a bit of a challenge to reach out and say, “Hey, my name is Carlos and want to connect.” And that’s something that we’re kind of trying to work through. And what are the other things that I thought? Again, I’ll reference SQL Cruise which is now Tech Outbound. I think one of the challenges they had was just being on a cruise itself. A lot of people that I talked into, I would talk about it and then they’ll like be, “Well, it’s on a cruise.” And then they do like, “Nope.”

Steve: Oh yeah. I remember hearing that on many occasions like where I wanted to go and pitch it with the company I worked for at that point in time and they are like, “What? You want us to pay for a cruise?”

Carlos: Right, exactly, right? So they think there’s a bit of a challenge there but some of the concepts were really good. One of the things that I miss or I lack about the SQL Cruise experience was that ability to kind of give back, to give my own ideas or thoughts around some of the topics. And so the rise of the unconference has played into this idea and I actually really like the premise of this idea of, ok we’ll come up with a track so here are the things that we can talk about. But then you know what we’re actually going to decide what we do talk about when we get there and then play it accordingly. And so that idea resonated really strongly with me and so that was what we want to put together.

Steve: Yeah, and I think that, I mean we went in that direction and I think that really is what we ended up putting together.

Carlos: Yeah, so I guess maybe let’s just jump to those outcomes. I feel like we were able to meet our objectives. We’ll talk about the challenges. We ended up having as we had 8 attendees, we had 5 speakers, myself and my wife, so there’s 15 of us in total. And there was some concern about the size and we went, we kind of had this thug of war as to whether we were or weren’t going to have it. And then really it came down to the speakers. I reached to Kevin, Jonathan, Randolph and Tracy, and I said, “Look guys, this is where we are. Do you still want to do this thing?” And they said, yes, and I’m like, “Ok, let’s do it.”

Steve: Well, and that’s one of those were I think to speaking of reaching out speakers. I mean, I was originally on the list as one of the speakers there. And in doing that I’d plan on attending but then when we got to the point where we’re sort of making the go, no go on whether we’re going to proceed with the conference or not it came down to just the overall amount of workload we had combined with the cost in getting there – coast to coast flight. That’s when we realized that we don’t have enough people showing up to be able to break even on it so that’s when I decided to opt out at that point and hopefully next time we can be in a position that it makes sense for me to be there.

Carlos: Yes, so that’s challenges from us as partners, only having two people hold down the fort. We have a project that needed some attention so that’s also a role. And then that week we ended up having two corruption issues with companies that we had not previously done work with and so that made for a very very challenging schedule.

Steve: Yup. So Wednesday, that day of the conference, that first day of the conference, that was the day that I spent pretty much the entire day fixing corruption for new client. And it was one of those things that had we both in there and hadn’t anyone left to cover anything. We would just have to say sorry we can’t do that we’ll have to help you another time would not been a good answer for that client.

Carlos: Yeah, that’s right. So that’s still something that we have to figure out from our end. But I play it to the reason for only having one of us there. But I’ll just go back to the objectives for a second. We felt like we were able to create that intimate environment where people could participate, get to know one another, and I think there were two major reasons for that. One, again because of our numbers we ended sitting in kind of a horseshoe shape, the traditional U. Attendees really like this because when someone started speaking even if that was a speaker which at the beginning it was mostly just speakers talking, so all the speakers were in there. They for the most part knew each other or at least of each other so there were some relationships already there. When people started speaking you don’t have to turn around to engage them. A slight tilt to the head you’re looking at that person and you could engage from there, so people really really like that idea. And then the other was the way we actually start the conference is that we started the conference with the four rules and then we did the three questions. So the four rules, some ground rules as to what you can expect out of the conference. And if you don’t have those things just speak up and then the next portion is just introducing yourself and actually having some questions to go off of. They are open ended to invite some discussion and to start making some connections. You can find who your people are. I think the combination of those two things helped to increase that engagement. Again, with the speakers being there, and making that commitment to be there for the two days we felt like we could make our objectives so that’s ultimately we went through with it. The other thing there and I think from an objective perspective is that the attendees were able to get to know each other, and we had great content. Right, from all of those things we felt like the objectives got met. I guess we didn’t feel like we were promising more than that and that’s why we were able to deliver. I was little bit nervous and I told the attendees beforehand that we’re going to have a small group. And even at 60 people that’s not a ton of people. I mean, when you think about other conferences that’s very very small comparison. But we’re going to be a smaller group and at first they were kind of like, “Huh, this is a little bit strange. I thought we had a few more.” But by the end I actually had two people say, “Hey this is right sized”, and we’ve actually again from a future perspective we’re going to actually come back off of that 60 number and it looks like maybe 30 is our new target. And again, we want everyone to be able to sit in that horseshoe shape for people to face each other basically with a desk and that kind of factors into that idea.

Steve: So you’re not just a face in the crowd you’re actually there participating and I think that’s the difference. And so then that’s was the other piece participating, so one of the ideas. We had a mix of traditional conference and unconference. So we had the speakers they each had a dedicated session that’s what we had promoted on the website as well. These are the things that they are going to talk about but we spice up that a bit and we actually solicited input and we said, “Hey, what it is that you guys want to talk about?” Everybody wrote those down on index cards. We put them up on the wall ad we actually had everyone vote on what they want to talk about. And one of the things that made it up unto the list was SSIS.

Steve: Interesting because that wasn’t even one of the things that we had anticipated that people would want to hear about.

Carlos: Exactly and so this was interesting for two reasons. It stemmed the discussion into two ways. One, so the card you set SSIS. We started with some questions and then we asked who wanted to talk about it, and one of the attendees, John. I think we mentioned him on last week’s podcast. John actually said hey I’d like to talk about that. So he and Doug actually lead that discussion and this is on Day 2 in the afternoon and because we already have that time together everyone had gone familiar with each other and so we were able to have, I don’t want to call it impromptu. It wasn’t as polish if you will again as a traditional session but it was very specific to what people wanted to talk about. Doug happen to have a VM, he pulled up his packages and we were able to do some demonstration and actually talk through, here’s what I do, here’s what we do and there were conversations all over the place and I learned some things just about logging that I didn’t know about. And then that turned into a discussion on [term unclear – 26:15] During all of that to which Randolph was like, oh I impressed. He hadn’t really seen it and he mentioned I don’t know that I would have spent as much time. Maybe it was only like a 30-minute discussion, right. But I don’t know I will spent 30 minutes on [term unclear – 26:32] But now that I’ve been exposed to it and we had in that context I fell like that’s something that I want to learn a bit more about. And so I really like that component of it. It was a very organic conversation that we didn’t have to worry about time necessarily and we kind of go with what people want to talk about.

Steve: Yup. Ok, so a lot of good outcomes there. As far as challenges what were the biggest challenges?

Carlos: Yes, so challenges, I think in the beginning we didn’t do a great job of getting user feedback. I think I know originally we kind of started with the idea of the lone DBA which I’m not opposed to. I think we knew we were looking for a segment of people we wanted people to easily identify whether or not they would come to the conference. And we have the lone DBA might be a good way to do that. Getting a hold of those people was a bit of a challenge and then even you companeros, right? We asked for feedback time and time again and we got not as much as we would have liked. You could also say, well, the way that we went about it wasn’t good. There were things to learn there. It felt like there was enough enough interest but we didn’t do a great job of connecting the dots and saying, “Hey, what it is that you guys really want to do and would be willing to pay for?”

Steve: And I think part of that, I mean, early on we were trying to get feedback there and I think that we just didn’t have the right venue for people to be able to present that feedback to us I think.

Carlos: You know, exactly that’s right. A lot of that initial feedback was I was contacting people directly and asking them, and I was reaching out to managers. I was very concerned about this idea of getting approval for people to come. And so I wanted to make sure that I could tear down from, and so when the manager looks at the website all their boxes would be checked. If they employee would like to come then they would get approval. And so from that perspective I think we did ok. I think maybe with that last mile of who should attend and why? I also think so having it in October so we knew that we were not Summit, and we are not Ignite, and we’re not you know. That’s not what we’re putting on even if our capacity 60 people we were not putting on that type of conference.

Steve: Right, that’s a very different type of conference what we’re going for.

Carlos: Oh yeah, exactly. However, having said that I don’t think that we fully realize the pull at least from the people in our network, right? So the people who are listeners, those are the two kind of we’re reaching out to, so speakers, SQL Saturday people as well. I don’t think that we had good data or the data that we had was wrong basically. We heard a lot of people saying, “Oh, I’m not going to Summit because it’s over Halloween.” We thought, ok, well maybe this is the year to try it. Even it’s not the same, we’re not saying it’s the same but let’s try to do something different. And I think the pull of those bigger conferences was just too big and lot of people ended up going to either one of them, with Ignite being the week before and then Summit being a couple of weeks later. The plans that they had in April and May changed and people ended up going or other things happened.

Steve: Right, and I think that’s a great sort of learning thing from being our first year doing it. How can we anticipate that back in April or May when we’re doing a lot of the planning? But I think now we’ve learned.

Carlos: And I think that, yes we talk a little bit about marketing, and so the other thing there is just a standardization. I think people have gotten very very comfortable with the way PASS us the events. Yeah, if you’re in that world already I think change is not, people seem resistant to change a little bit there. But I think one of the nice things is that we’ve had the event we can now have people, in respond to it give feedback on it. You can see it. Jonathan wrote up a very very nice post about it. You know, the speakers have come, they’ve seen it as well and so hopefully some of that will filter into the community.

Steve: The other challenge that we need to talk any about the financial side of it.

Carlos: Yeah, so that’s right. At the end of the day, a slightly different, so I guess this kind of goes into this standardization process. I’ll take a SQL Saturday haven’t been involved with those and putting those on particularly that it’s a free event. Because of the standardization that PASS has done, so PASS is sponsoring, Microsoft is now sponsoring, and you have all these vendors. It’s kind of a known quantity because they’ve put all that time and energy into explaining what it is. Having people go and whatnot. That from a sponsorship perspective, the entire event can run off of sponsorships and then lot of volunteers and all these kinds of things. So because we’re going with the SQL track model, we knew that sponsorship would be very very difficult because there wasn’t a traditional, hey you get to have a booth, talk to the attendees, raffle tickets, things like that. Let alone from the numbers perspective. And so that was a challenging idea as well. And so as a result we depended entirely on revenue from the attendee tickets. It ended up being about $4,000, our expenses were about $6,000, and that was another thing that’s bad decision. I knew that when we made that decision for Steve not to come out and to move ahead with it that we couldn’t cheat the people who had come and basically budget it, right? Because I didn’t want them to feel like they were getting short ended. I wanted to make sure that we were meeting the commitments we’ve made to them. But from a business perspective so as an entrepreneur you can’t contain to do that all that time and energy and then come out $2,000 in the whole doesn’t equate to long term success.
Steve: Now, if we were like a software development company and we have a whole bunch of products that were being sold out there and we could ride off that loss as an advertising expense perhaps or something, then yeah we can keep doing that indefinitely. But I think we really don’t have that. I mean, other than our consulting business we don’t have that other revenue stream.

Carlos: Or outside investment, things like that.

Steve: Yup, yup, so I think being able to set it up in way that it works so we don’t lose money to do it I think is a key factor to make this sustainable going into the future.

Carlos: Yeah, and so I guess we can talk about that future and I feel like we were pretty close. I mean, it’s not like we were at the Ritz-Carlton and doing all kinds of really crazy over the top things. Although we went on the cruise, we had a nice buffet dinner that was very very nice. We had plenty of food, the room was nice and all these things. It wasn’t over the top I think if we got from 8-16 we would have been in a better place. So I guess talking about the future, so a couple of takeaways. I guess another challenge. I knew it would be slight challenge to use the word companero. I don’t think I understood how challenging that was going to be.

Steve: And that’s one of those things that in retrospective it’s easy to see that now. I think just looking back at our podcast Episode 100 with Kevin slaughtering the name companero. Comp… however you say it in the beginning. I mean, maybe that should have been a clue but we kind of have fun with it. But yeah, I think that’s good point. It’s a tough one to say or even to spell for a lot of people. Even me.

Carlos: I don’t really like the word conference, right, particularly because we’re going for the unconference idea. I’d like to stay away from that word conference. And so some of the feedback that we got, some of the thoughts that we have about potentially putting on if we were to do an event in 2018, some of the things that we thought about. So one, renaming it to SQL Trail. SQL Trail, much easier, it’s kind of already part of our brand as well, right. “We’ll see you on the SQL trail”, we’ve said that on every episode since Episode 0. That has been part of the podcast. And so I feel like incorporating that would be a bit easier.

Steve: Oh yeah. And I’d be curious to hear what people think about that. I mean, does SQL Trail make more sense than Companero Conference? We’re always looking for feedback and those kind of things.

Carlos: That’s right. The reason I like it even and again kind of using, even SQL Cruise or SQL Saturday, they are not using that word conference. They have that short name that can be used to identify what it is without using the word conference. And so that’s what I would like to do, and so SQL Trail would be a potential.

Steve: Yup, I like it.

Carlos: The other change that we were looking for too going forward is we had the event for two days. What we did is we use, and it only ended up being about an hour but we did an hour of introduction. So maybe an hour and a half in the introduction, and then the session selections for the Thursday. Part of the feedback was, hey, let’s push that to either before the conference basically or separate it from the first day of the “conference”. And so what we were thinking of doing is making that 2½ days. So we will start for example on a Wednesday, at 2PM. You have the welcome, everybody comes, introduces themselves. You know, introducing that idea of the social into the start of the conference and then choosing sessions that afternoon and then having a dinner. Right, so then at the end of the day on Wednesday, you know the sessions, the schedule for Thursday and Friday. And then on Thursday and Friday you come and it’s more, I won’t say traditional but from a schedule perspective you kind of at least know of what to expect at that point. And then if people are travelling or whatnot, again they can come in the morning for whatever reason they are a little bit late. That component of it is taken care of. And then the other thing that people really really, everybody talked about was they’d like more hands on and we did talk about a hands on. Well, nobody criticize I guess I could see that being one thing that we kind of tweaked a little bit from our original session. The hands on, that lab idea sounds really nice but it is a lot of work. It is super hard to pull off well because you’ve been in the lab, some people breathes through it, others have no idea, some are in the middle. How do you work with all of that?

Steve: And do people bring their own computers or do we supply computers?

Carlos: Yeah, exactly, again if we are supplying computers and all of a sudden it’s much more expensive, and logistics and things.

Steve: And if everybody brings their own then we got to figure out how we get the right software installed.

Carlos: Yeah, exactly or are we installing software things like that, all those kinds of questions. So one of the thoughts that we have and we’re looking for the right people is to that might be a way to partner with our sponsors and to have a couple of things that we say ok in the conference you’re going to be able to come away with. Let’s just take as an example, you’re going to build an availability group, create a container in SQL Server, create an SSIS package and use some DMVs or do some database monitoring. You got to get hands on on those four areas and then have the sponsors come in and help support those things and again I’m just using an example because I’m talking to them. Like the folks at [term unclear – 40:41] Having them come out, they can help with the lab in setting up the how to, this is how I setup my container in SQL Server. They will make that commitment but then they will also be able to then say, ok you’ve seen how that’s happened. Now here’s how we can help make your life easier.

Steve: Yup. You know what I think that would be incredibly be valuable because so often when you’re at the conference and you meet with a sponsor or a vendor and you talk to them and you think, “Wow, you’re product or your stuff seems really cool.” But then you get home then you realized, “Oh, how do I try that out?” or “Where do I go now that I want to try it out”, and you realized of there is not trial to download or I got to talk to a marketing person or something like that. But if you could just have like hands on demo with more than just here’s a lecture from a vendor. Like, here’s how we actually make it work hands on, that would be awesome.

Carlos: But in conjunction also with, ok, so here’s how you could do like SSIS. Let’s build an SSIS package and some basic things. Now here is how you could make that easier. And those conversations and if the sponsors are willing to commit to come to the conference and spend the time to make those relationships then I think it will be easier to have those conversations when it comes for lab time towards the end of the conference.

Steve: So we should be thinking about what sponsors would be interested in doing that.

Carlos: That’s right, and so companeros if you have ideas let us know. And so this kind of gets us to the question of should we try this again in 2018? I can say that I had a blast, right? I enjoyed it. I enjoyed getting to know Erin and Gretchen and Bryce and Jeff and Dave. Good thing we gave him a companero shoutout the other day. Getting to know those guys and of course the speakers in a different way. But ultimately I think we want to see if this has power. Is this something of interest? We’d like to hear from you. I think you can do that in a couple of ways, right? Obviously, social media is one. The other is we’ll make available on the show notes page. We’ll put in a little section. I don’t if poll is the right word. But we will give you the ability to sign up, to put your email address in and say, “Yes, I’m interested in being part of the conversation about the 2018 conference.”

Steve: Yup, and we will not add your name to any kind of spamming list. We will just use that for discussion of where we want to go, or what we want to do, those kind of things.

Carlos: That’s right, and many of you have trusted with your email address and we hope that you’re not seeing that as, you know, we are providing good content and obviously there is a subscribe button there and everything which you don’t want to. But I think if we can get to, and I’m not sure what the right number is. But I think if we can get a certain number of people to sign up to say, “Yes, I’m interested in providing feedback at a minimum.” And would be interested in being contacted or being informed about when and if we do this in 2018; if we can get a certain number that I think that will be one of the criteria that we’ll use to determine if we should go forward.

Steve: Yup, I like it.

Carlos: Yeah. So I think that’s the retrospective. I think we learned a lot. I think I’m glad that we went through it and thanks obviously to those who attended and to the speakers. I guess I should say in that number, so we talked about those numbers, those expenses. The only thing I provided for the speakers was the ability to attend, obviously their food, and the cruise, a t-shirt and a notebook. Randolph came in from Canada, just all these flights, hotels, all of it was on their own. So we can’t say enough thank you to them. If we had those additional cost obviously it just wouldn’t have happened.

Steve: Yup, so what are we doing next?

Carlos: Yes. I guess with that we’re testing out this idea of the SQL Trail. We mentioned this event to talk about at the end. So Steve and I thought, again kind of in that idea of engaging with you companeros and continuing the conversation, we are putting together an event at Summit. So this event is going to take place on Wednesday evening. So we’re actually going to bundle it up. It’s going to start in the middle of the last session of the day on Wednesday. I don’t know we go to 7PM. We’ll put out the details, again, this idea of SQL Trail. I was thinking about, you know some SQL Trail should that be something that kind of call our defense. This will be included on that. We’re going to provide some food, it will be at the, which I also can’t remember the name. But we will provide advertisers and things and we’ll have raffles for drinks and whatnot which we will explain later. But I was thinking, you know what, SQL Trail and then I was thinking about food and my wife actually said you should call it Trail Mix.

Steve: Ah, so it’s the Trail Mix event.

Carlos: There you go, that’s right. It’s the SQL Trail Mix event. So that’s what we’re going to call it and we’ll hope you’ll come.

Steve: And how can people find out more about it?

Carlos: So it will be on the shownotes. If you’re on our mailing list you’ll get an announcement about that. And then through social media we will be publicizing that as it comes around. We’ll invite you to sign up for Eventbrite so that we know approximately how many to expect and again all the other details will be there as well.

Steve: But even though it’s through Eventbrite there’s no admission fee or anything like that. It’s just trying to get people the RSVP that way.

Carlos: Yeah, exactly so just we know and ultimately it will help us with the raffle for example because we’re going to raffle up some of the SQL companero shirts. And the notebooks and things and so that would just the easy way we have your name. We will preprint them and then we can just pull it out of a hat and go from there.

Steve: Yup, awesome.

Carlos: Ok. Well, I guess that’s going to do it for today’s episode. If you have other questions obviously reach out to us. We would love to continue to talk with you. Our music for SQL Server in the News is by Mansardian used under Creative Comments. You can always reach out to us on social media or connect with us on LinkedIn. I’m @carloslchacon.

Steve: Or you can connect with me on Linked @stevestedman and we’ll see you on the SQL Trail or at PASS Summit at SQL Trail Mix.

Episode 114: How do you start consulting?

One of the advantages to a small conference is the ability to take attendee feedback and put it in place during the conference.  We actually made time for the attendees to pick topics they wanted to discuss and this episodes comes from one of our attendees Aaron Hayes.

How do I start consulting?It can be very tempting to think of the good life of consulting.  I almost liken it to playing the lottery–what am I going to do with all that time and money?  While the odds on successful consulting are a bit higher than the lottery, just saying you are a consultant won’t automatically bring in the clients.

The reasons data professionals get into consulting are varied and in this episode we are joined by Randolph West and Jonathan Stewart, former podcast guests, to talk about the reasons we started consulting and some of the challenges along with our decision.

From my own personal experience, working for yourself–whether you consider yourself a contractor or a consultant–is very rewarding but demanding work.  There is no one to tell you want to do, but there are very few security nets as well.  One of the most important ideas I can suggest for those who want to own their own business is–forget the technology, how are you going to help other people?  If you are ok with the idea of focusing on others, then there are great opportunities in store.

Do you have thoughts about jumping into technology?  Let us know in the comments below.

 Episode Quotes

“I hate the lost productivity involved in sitting at a desk all day.”

“Becoming a consultant allowed me to… choose my customers, choose what I want to work on.”

“You can do other things but be known for something”

“word of mouth turned out to be a very bad strategy.”

“A consultant is I’m helping you solve a problem, a contractor is getting some job done for x number of hours.”

Listen to Learn

10:26 Challenges in consulting
15:45 Having your own branding or niche as a consultant
19:08 Difference between a contractor and consultant
42:57 How consultants find their particular customers? What are some of the strategies?

Transcription: How do you start consulting?

Carlos: Companeros, welcome to Episode 114. It’s good to have you on the SQL trail again.

Steve: Carlos, Episode 114, sounds like a fun one.

Carlos: Yes, so we are piggybacking a little bit off of a conversation we had at the Companero Conference now that we are on the other side of it and recording sessions afterwards. One of the interesting things that we did at the conference that I’d like to continue and we’ll talk about more of this in our post mortem, but the idea was we wanted to leave some time or some unstructured time where attendees could submit topics that they wanted to talk about. Which we did, and then the attendees voted on the topic, so everybody got to contribute and then we voted. One of the topics that came up was this idea of “How did you get started in consulting?” And so we recorded this, we have it during the conference and then we thought we would make it available to you the podcast.

Steve: And this one was suggested by Aaron Hayes.

Carlos: Yeah, so Aaron of Chicago, shout out to him. Thanks Aaron for coming.

Steve: Speaking of shoutouts do we have other companero shoutouts, Carlos?

Carlos: We do, I guess so keeping with the conference trend so John Szewczak–I think I’m saying that right. I spent two days with him. John, if I didn’t get that last name right, I apologize. This was an interesting event that developed as well in that some of the other conferences that we go to and again the event was two days, so you get to know people. We were heavy on interaction and one of the other sessions that was developed was SSIS, so John who was an attendee had been doing quite a bit in SSIS and as the conversation had jumped up he had responded to a couple of things so we actually had this session. We actually invited Doug who had given a session previously and John up and they led that discussion on SSIS. There were no slides. It literally started with a couple of questions and they just started bringing up a console and just kind of, “Hey, this is what we do in giving thoughts and experiences.” And I thought it was very very cool that contribution can come from anywhere. If you are following the Twitter notice, and we will put it up on the show notes episode for today but you can see a picture of John up there leading that session.

Steve: Well, maybe next time when we do our call for speakers we should open it up to the previous attendees first.

Carlos: That’s right. In fact, he did say afterwards that he would be interested in coming back and speaking, which we are going to make some changes there to that whole process as well so it would be interesting. Another shout out there, way too many to mention, but this week I attended our local high school for career day. And I’ve actually done this for a couple of years now, so I lived in a very blue-collar area. You can take that as you see it. So lot of families, so dual income, and the majority of the folks in our area are not college educated, so that’s kind of how I am looking at that. The high schools they have a lot of trouble from a demographics perspective. We tend to be on the lower end of the socio-economic scale. This idea of kind of being able to break out and, these kids have an uphill road. Anyway, I was there and we’ve been talking and I wanted to get across or extend this idea of the power of the network and why making connections is important, and then connecting with the right people, and there is an investment in that. We’ve talked a little bit about that on this program on the podcast before. And so what I did, it was kind of through the moment. It wasn’t as planned as what I would have liked to but I went ahead and took a selfie with the class. I posted it up on Twitter and said, “Hey, this is what I’m doing.” I put in the #sqlfamily, and then said, “Please reply, like or tweet”, to let people know kind of that power of community and where it goes, and it was super interesting. Obviously lots of comments came in but then just seeing I think the breadth. So while I was at the school, I was at the school for 5 hours and while I was there and then the tweet continue to get likes and whatnot afterwards. It went as far as Brazil, people commenting, which they thought kind of interesting and then all the various states, and so we had several people just reply, some of the people retweeted and then of course liking. And you kind of do some analytics on that tweet and they were able to see, again, that idea of so of my followers, which I think I have around 700. It went way beyond my own network, right? But it was because people knew who I was and I was asking for something that was really easy to do and they could feel like they were helping. Yeah, it was an interesting little experience.

Steve: Ok, really cool, so I guess the thanks goes out to all of your followers who retweeted and all of their followers who come in and retweeted it as well.

Carlos: That’s right, who connected and got in on that. I really appreciate it.

Steve: Interesting experiment there.

Carlos: That’s right. Ok, time for a little SQL Server in the news. There are a couple of more announcements for Ignite that we haven’t quite gotten through but this one I thought was interesting, and we’ll see how it goes. My understanding is that it’s currently in preview, and we’ve talked about Azure data factory on the program before. Although it was really in its infancy and has definitely gained traction a bit now, but they are now allowing to deploy SQL Server integration packages to Azure. There are some parameter around it but the Azure data factory is ultimately what’s going to be powering that and there are some limitations as to what you can connect to and whatnot. But that idea of being able to get SSIS in Azure is picking up some momentum and I think they will continue to work on it and it will be a little bit more usable for those who are more familiar with the GUI.

Steve: So perhaps it’s time for us to do an episode at some point in the future on SSIS in Azure and get someone who knows a lot about that.

Carlos: That’s right, that’s right. Which interestingly enough we are getting ready for summit, and so that is the time that I like to go out and make connections to some of those program managers, and the MVPs and whatnot. And so I will definitely put them on the list of people to reach out to. We should probably go ahead and mention, we don’t have all the details yet, but we could probably go ahead and mention what we’re trying to do at summit.

Steve: Yes. Well, we’re both are going to be there and we’re trying to sort of get an after hours, some kind of a gathering together. Still working through the details on that right now but a companero and podcast listener get together where we can just hang out and chat for a while.

Carlos: That’s right, do kind of a happy hour type of thing. Again, those details are still forthcoming, however, I will give you a little sneak peek and that is it will be to your advantage to leave a review on iTunes. So we are going to work out some incentives. Those of you who’ve already done that, thank you. If you haven’t we invite you to go over through iTunes and leave a comment or review. That will be key to, not key, obviously you could come out. We’ll do something for those who would love feedback.

Steve: So if somebody wants to attend that after hours drinks, whatever we are calling the event, how can they reach out to us?

Carlos: Yes, so obviously through social media, you can hit us up on email as well. So probably I guess through Tweeter or LinkedIn. It’s probably going to be the easiest way. One of those two I’m assuming you’re on that social media. We’ll make the information available and we’ll put it up on our show notes page for today’s episode and future episodes. Today’s show notes episode is going to be at sqldatapartners.com/consulting

Steve: Or at sqldatapartners.com/114 for the episode number.

Carlos: Yeah, and so that would probably be the best way to get that information and we promise for next week’s episode to have that.

Steve: Alright, excellent.

Carlos: So now that I’m thinking about it I should go ahead and plug, so at sqldatapartners.com/podcasts, you do have the ability to join our newsletter and we can make sure that it goes out through those channels as well. Ok, well let’s go ahead and get into the conversation here.

Carlos: We’re actually here at Companero Conference. One of the sessions that came up, one of the interesting things that people wanted to talk about was a little bit of the question of how we got into consulting, so we have Randolph and Jonathan here, and so we are going to have a little brief panel discussion and talk a little bit about it, and then maybe open it up for questions or how we decided to go. First, we will let you give your, the brief story of why you decided to become a consultant.

Randolph: I become a consultant because I hate working full-time. I hate the lost of productivity involved in sitting at a desk all day hearing people have arbitrary conversations that have nothing to do with work and not getting their work done. I hate the idea of a 9:00 to 5:00 or an 8:00 to 5:00 shift where you are expected to be sitting at a desk and looking at a screen or doing stuff when your productivity might be better later and during the day or even at night in my case. I found myself to be far more productive just working from home or finding a quiet space as just as shared workspace that’s in Calgary that I sometimes go to. And I can get more done in an hour of quiet time than I used to be able to do in a full day of work working full time. Plus I get to choose my customers. I get to work as much as I like or want to and I get to travel.

Jonathan: I’ll give a slightly more PC answer than Randolph gave. There was a lot of hate in there.

Randolph: Intense dislike.

Jonathan: Ok, sound guy, you fix it. Let’s see you edit that. I became a consultant because lot of the same reasons that Randolph said too for me. I hate to not be thinking. I know right, I intensely dislike to not be thinking so one of the things that happened a lot of places that you will do a project and a project will be great, will be interesting. You will do a lot of work and stuff too and then it sees lulls. In those lulls sitting at my desk, you know, you can only read the internet for so long and then you start getting bored. And when I got bored, it’s just really bad when I got bored because I don’t like to not be thinking. I’m always thinking, and when I can’t control what I’m thinking about then I begin to think about other things too like, “How much I’m getting paid?” There are all kinds of other things that would happen when I wasn’t thinking. So for me, becoming a consultant allowed me to, as Randolph said, choose my customers, choose what I want to work on, be in a load up as much as I want, be a tech time when I want because I want to do work, I can do work. I do work in the middle of the night because I don’t really sleep a lot, so it let’s me add work to it and lets me choose the things I want to work on. As technology moves forward I could move forward with it and not be stuck based off of a company wants to go forward with a new product or not. I can choose what I want to work on. I can choose if I want to move more toward predictive analytics or if I want to move more towards visualization. It’s my choice so it allowed me to control my career and be happy with it because literally if I don’t like the work that I’m doing that’s my fault because I chose what I want to do. So that’s the short story of how I got into consulting.

Carlos: I told the story a bit before. It’s always been my desire to be an entrepreneur, the idea is very sexy. We are Americans, right, we build stuff. We make our own things.

Carlos: My first company was actually like a sort of computer repair. My partner is Jeff Morrison, called it Mor-Tech. he kind of started it and I wanted to help him and then that kind of fizzled out because, anyway, it just didn’t work. You are always afraid, you are afraid to make that leap particularly in technology. Like, what if somebody asks me something I don’t know so that helped me back for a while. And then I was working for a law firm and I was in a position where I was seeing the contracts of these consultants that were coming in, and then you get to a point where they start coming to you for questions and you’re like, “Hmm…” Admittedly, some of them were in California so a bad basis. I was seeing California rates coming in and I said, “Well, gosh, I’ll do that. Maybe it’s time to make the leap.” The timing just happened to be such that my wife finally is going to give me the green light and so that’s kind of how we started. I think that was, and I looked back at that and while the desire was there I had some semblance of what I wanted to do or kind of seen what was there. I did not have a great footing or foundation of what it meant to be a business owner, and I wished I would have waited, not waited but maybe did things a little bit differently there initially.

Carlos: Ok, so Steve, do you want to give us, how did you get into consulting?

Steve: Well, I got into it a couple of times. The first time I wanted to try being a consultant and this was 12/13 years ago now, maybe. I sort of quit my job, hang out a shingle, started talking to people and my previous employer ended up hiring me. I ended up doing work for them. It was really kind of a weird consulting scenario because I ended up working for them almost exclusively for 1½ year or 2 years.

Carlos: After you had been a full time employee.

Steve: Right, right, and then when that gig came to an end that was when I kind of learn a little bit about making sure that you don’t put all your eggs in one basket which is one client. And I made rules for consulting from there on, and I broke them a few a few time, but then I ended up getting a couple other gigs with other clients and then ended up going full time with one of those clients as an employee which took me out of the consulting ring. And then about 2½ years ago it was when I got back into it. So where I left that job and went full time. I mean, for me, it was really. I mean, just jumping back in and wanting to work with different clients rather than the day job. And I think that, I mean, enjoy being a consultant because it changes the dynamics of what you do every day. I mean today, any day, I usually get up and started early and I work early and oftentimes by 9AM. I’ve completed more work for clients than I ever would have in a full time job by noon, or maybe by the end of the day. And it’s just the nature of consulting that allows you to do that.

Carlos: Right, right. It is interesting that I think a lot of clients or a lot of consultants will start that way. That was my experience as well with a former employer and you get in to it for whatever reason and those rose colored glasses helped you jumped in, and then there comes a point where you’re like, “Holy cow! What did I just do?”

Steve: Oh yeah, absolutely.

Carlos: I think from there let’s go in and I guess talk about, let’s do challenges first, so challenges of consulting. So you mentioned lulls, you know, there’s lulls at work and I think there is definitely lulls in consulting, right? So what are the challenges that you find being a consultant?

Randolph: Scope Creep, so you sign up for a contract that’s going to be three weeks, four weeks, and then suddenly you’re working 6 months. It’s not a bad thing but it might be not the work that you thought you were going to be doing. That’s the good side of challenges that face you. The bad side is where you’re not working for 6 months at all which happened to me two years ago. And that’s why I got more into acting and making films because I had all the spare time but the challenges that I had to pay for it somehow. I found myself compromising; I only do performance tuning to start doing development again because I used to be a developer, so I had to get back into that and I was rusty. So the challenge is that I wasn’t be able to do the work that I wanted to do but at the same time I was keeping my old skills fresh and learning new stuff as well. I was like upside there and that’s kind of how I got involved with a little company called SQL Data Partners. You may have heard of them. Good bunch of guys, the one is a bit odd, then there is Carlos. Hi Steve! I’m kidding, Carlos is the weird one. The challenge is I didn’t work for six months, that was rough. I won’t lie.

Jonathan: Since they’ve already touch on those challenges too, I’ll add another challenge too because we work for ourselves. Another challenge too is something that I didn’t realized until I was doing it is, people don’t like to pay you when they said they are going to pay you. One of the hardest things about making a jump into independent consulting is understanding that for that first month, two, three, depending who your first client is. You may not get paid. You get paid but you get paid down the line.

Carlos: When they want to pay you.

Jonathan: So not just getting paid and being prepared for that and then to understanding how invoicing works and stuffs too. All of my clients are really, they’ve helped me invoice them because there is a lot of things setup invoice, track your time, and stuff like that too. But at the end of the day, you know, been able to go through and invoice somebody and then able to change the invoice in the fly, something happens and stuff too. For me that was a struggle because that wasn’t my strength. Accounting wasn’t really my strength so I actually hired an accountant. Let them do that. So there is a lot of ways to be handed on your business and stuff there too. Some people do it all by themselves and I commend them because that’s a challenge. But there is a help out there so don’t be overwhelmed. You can even contact me. I’m sure Randolph and Carlos as well, if you got questions you want to do it. I see your card, Randolph. You can be Randolph, and I can be … I’ll make a normal $1 bet, we’ll showing our age. But no, the challenge is though is there is other challenge too besides just getting the work and stuff as well, it’s the stuff behind the scenes. The thing is that from taxes and stuff like that and understanding the things outside of your normal day to day task. Those are some of the small things that come being an independent consultant.

Carlos: Yes, as you mentioned tasks so as an employee unless you get to Kevin’s level, right, or until you get to Kevin’s level. I won’t say they are not micromanaging but there is a certain level of expectation, so as database professionals that obvious that you’ve been given is to take care of the database. Maybe they are not giving you every single thing you should do but your tasks is there and assigned. Once you then kind of make the leap, there is nobody giving you that task anymore. The expectations are we want, “make us happy”, and how do you that? The other struggle there is what should your tasks really be? When you first start, you think, you kind of working as an employee like you do anything. So performance tuning, you write a report, you go and talk to whoever about there connecting to a database, right? Lots of the breadths of your assignments is very wide. Now as a consultant it’s slightly different because of that idea of what value are you bringing? Initially I was like, “Well I am a database person you can hire me to do any database problem.” “I don’t have a database problem.” And you’re like, “Wait a second.”

Randolph: Let me find one.

Carlos: Yeah, exactly. I think coming up and actually kind of narrowing the scope a little bit or defining, having a well defined set of what it is that you want to do is very difficult because you feel like all of a sudden, “Oh my world is shrinking a little bit.” But it’s one of those things were you have to go down. I think of Alice, when she takes the potion and shrinks down, you kind of have to shrink. You think about, you could apply this to, when you think for example David Klee. What do you think about?

Randolph: VM guy.

Carlos: VM, VM work, right? When you think about Pinal Dave, what do you think about? Performance, right?

Randolph: One blog post a day for 10 years, that’s what I think of Pinal Dave.

Carlos: You know, when you think even of, so Patrick LeBlanc, or some of these other folks. There is one thing that they are kind of known for. Do they get to do a lot of other things? Sure, but they have that one little niche. Or Jonathan Stuart, what do you think about?

Randolph: Visualization.

Jonathan: My hair. But no, that’s actually the point. That’s one of this I was thinking about too is, and it actually goes falls right align the things. So Randolph was talking about he wants to do performance tuning, so as you figure out what you want to do become known for something. Brand yourself. Your branding is huge. So I chose my hair to make my logo as my branding, but even still visualization is my thing although I do data warehousing and all kind of other stuff as well. You have that niche that gets you in the door and then you can do other things. But be known for something and then you can do other things as well because what happens is that if you’re just a generalist nobody thinks of you specifically to do that one thing when that comes up. If you want to do that one thing, be known for that one thing and then, “Oh hey, we have X, Y, Z as well.” You may not be able to do X, Y, Z but then I can call Kevin, I can call Randolph, I can call Carlos to help me out do those other things. But that gets you in the door. Be known for that one thing, be very good at that one thing. While Klee is known for virtualization he does all kind of other stuff as well. But the virtualization has got him in the door because that’s what he is known for around the world. When you say Klee, you think of VM. VM ware has changed its name, Kevin say as KleeM ware because it’s what it has become. But that gets him in the door, that’s his brand. That’s what he is known for but he is phenomenal with all kind of other stuffs as well. Be known for that one thing, but be able to do other stud as well. Even with Randolph, like you said, he got to do performance tuning but he can do other stuff as well. So if performance tuning gets him in the door and he goes like, “Hey, I want to do X, Y, Z and expand to Azure SQL Database.” He can do all that as well. So once it gets him in the door he could say, “Hey, I had all these other services as well.” But that’s what got him in the door, that’s his brand, that’s what he is known for.

Steve: Right, so I think some of the challenges and I think some of the others mentioned this, things like scope creep. I mean, oftentimes you will start on a project that you think is going to be one thing. And then after meeting with the client and understanding what they want, it ends up growing into two, or three, or four or ten different things that you never even expected. And that could be good or bad depending on how you look at it. I mean, it’s good and it brings in more work but oftentimes it can be bad because one of my rules is never do more than, well hopefully more 30% of your time with any one client. Sometimes I’ll break that and go to 50%. But when the scope creep goes the point that they want you to be there as like all your time with one client, I just don’t go for that because that’s way too risky. Because when the relationship comes to an end with that client which it always does eventually then you’re sort of out of work at that point. I kind of fight that scope creep by pushing back and prioritizing things but then limiting my time amongst multiple clients. Which I guess sort of brings up one of the other challenges that have come across in consulting is, when you’re working part-time with teams that are working full-time, and oftentimes they kind of expect that you are on the same full time schedule they are, even though you are a part-time consultant, and trying to sort of balance between the different clients so that you can responsive as needed but also so you can spend the right amount of time with each client each week.

Guest: What is the difference between a contractor and a consultant?

Randolph: Well, a consultant I feel gets to choose what you want to do, a contractor gets told what to do. You still have billable hours and you still have to fill in a timesheet. It’s all the same thing. It’s just who you are working for is a little but more blurry because a contractor might work for a contracting house like I use to and then I get told, “You’re going here tomorrow. You’re going to work six hours, you’re going to bill eight.” Hey, I’m not going to lie. That’s the difference for me. A consultant is I’m helping you solve a problem, a contractor is getting some job done for x number of hours. The religious work comes in whether contractors are actually doing anything useful, and some are and some aren’t. It’s just depends who you’re with I guess.

Carlos: You think these goes to that idea of like of the branding? This idea of are you being paid for your hands or for your brain in a sense, right? So if you’re being, like brought in, here’s the work we want you to do. I tend to think of that as the contractor route. If it is, “Hey, I have this problem how do you think we should solve it?” Ok now, I’m in putting it into the process and again not a generalization but that’s my delineation. You feel good about that?

Jonathan: That’s exactly how I say it too because some people say that’s semantics is the same way but a contractor is like a staff person. You are filling a role, you are filling spot so they can just throw work at you. As Randolph said, you are being told what to do whereas the consultant is solve a problem, help me solve a problem. You know, like management and consultants, they are all helping solve problems whereas the contractor is somebody who is filling a seat for a specific time and a role.

Randolph: Yeah, there’s nothing wrong with doing contract work if that’s what you enjoy.

Carlos: Yeah, and I guess that’s an interesting point. It’s not like the one is necessarily better than the other. What are you constraints? What are your box? Where do you want to be and then that’s going to make your decision there for you.

[unclear conversations]

Jonathan: I used the same approach in regards to vendors. So we are going to do an imaging project and we sent RFPs out and have them to kind of do demos. I look for things like contractor versus consultant in the vendor aspect. Are they are going to help us solve problems or just put the imaging system in and leave. I think I can use it both ways in regards if you’re interested in consulting like myself. You can still see that in the vendors you deal with. One of the things too that a consultant has to worry about and a contractor doesn’t even though they can actually be the same person at the same time as well but the consultant has to work on managing relationships as well, if that’s at the end of the day you don’t want as Douglas is saying, you don’t want to just come in and fix something because that’s the end of the gig there. I want to develop a relationship so I would rather spent four hours with you every month than spend five hours with you one time upfront just to do one thing. That’s another big thing there too so.

Steve: I think that, I mean, as a consultant you are coming in to do something that sort of project or problem base where you are trying to come in and create something or fix something or come up with a solution, whereas, a contractor you are punching time on the time clock. And not that you’re not getting things done but you’re showing up and you’re there on the clock for whatever is needed. And I think the comment was, are you paid for your hands or your brain? And I think that that was a great analogy there in that, if you’re there just sort of on assignment for a certain amount of time and you will do whatever needed in that time, that’s kind of the contractor role. And when you’re there to help get the job done to focus on a specific task then I think that’s more on a consultant where you’re either building something or advising on something or coming up with solutions, and that consultant role is the piece I really like.

Carlos: Great, and again, so if you we did mention it, I should mention it here that neither is right nor wrong. The labels are not really important per se. I think it is however more of a philosophy of who’s kind of driving the boat in the relationship in the sense of who’s dictating what work will be done, and that can be a fine line sometimes. I think kind of going back to your comment about the expectations of the client dictates quite a bit about of what role you’re taking on.

Steve: Right. I think some of that comes from just what the client has been used to working with. I mean, if they’ve been used to working with a staff type position where they get a body and can work specific hours. That can be very different than getting in a consultant who’s there with a specific specialty like we have to solve a problem or fix something or help them with their performance whoever it may be.

Kevin: Kevin Feasel, professional podcast guest. You guys just talk about… it’s better than being an amateur. So you guys just talked about brand, about building up skillset but I would like to bring in the next question as somebody who has absolutely no interest in doing consulting work, hello employers, how do you go about finding the particular customers that you want to work with? How much of that is them finding you, how much of it is you finding them? What’s the strategy behind that?

Randolph: Most of my customers were from LinkedIn which surprised even me. I thought I’d find people through word of mouth, those were terrible customers because what happened is somebody says, “Oh, you know computers, I know somebody who could use your help.” And then you end up working on MySQL doing index tuning and then after 20 minutes you got no more work for them because… They are only useful in certain context. But the point is that word of mouth turned out to be a very bad strategy. LinkedIn where I specified this is what I can do, this is what my company does. People who need it usually go there because they are business peoples so they are looking for a problem solver for that particular problem. I do give my cards out that’s why I have them up here because if anybody hasn’t got one yet I’m going to give it out because I want people to associate me with the fact that I am available to do almost anything. Because the chances are 5% of the people call me back and out of those 1% or 2% of those 20% of those people will actually turn into a paying customer. So I have to go and do the whole mail shot or whatever so that I can get 1% back to actually pay me to do something. There is a lot of targeted advertising I could do but I don’t have the time or the money to do that right now, and I don’t want to because I don’t want to work full time.

Jonathan: Because if you work full time he will never be able to play dead in TV shows and movies. It’s acquired arts skill. Don’t you play dead with me sir. There is a wide gamut of ways that you can acquire customers. Most of my customers I do partnerships with a lot of people. A lot of people find me because I speak a lot so I go places and stuff too and people say, “Well, come up to …”, and I talk to them too. A lot of these are business owners. They have their own SQL practices as well. But it’s like, “Oh hey, we might want to partner with you. We want you to do business intelligence. We may want to do visualization. Can you help me with this client and do such and such.” Speaking is a great way to get your name out there, to be known for certain things. So for me, like I said, a lot of my clients come through not just word of mouth but from partnerships that people I’ve met on the speaking trail, or heard me on SQL Data Partners Podcast, right, I got people contacted me in that way as well. It’s not really hard to find me because I’m the guy with hair. See it’s a branding thing again, right, but have a brand. Everybody has a way to be branded you just haven’t thought about it yet. So if you really consider doing this, think of a way to brand yourself to get out there and then there’s multiple ways. As Randolph said, he gives his card to everybody. Anybody can give us cards because that’s a cheap way to advertise. You know, me, it’s not really a cheap way because I’m flying everywhere to do this. But there’s so many different ways. There is not just one way. You got to think outside the box. You know, and there’s more ways than we talk about. I’m sure Carlos has another way as well.

Kevin: Write a book.

Carlos: Initially we tend to suffer. It’s very similar to the idea of finding a job. How many of us has just taken a job that has come our way? Do we actually think about the industries that we want to work in? They type of people that we want to work with? The type of projects that we want to take? We might say that, “Oh yeah, I want to do migration”, so SQL Server 2017. At the conference several people have mentioned, “Well, they approach me with this job. I wasn’t really looking for it but it paid me x percent more than I was making”, and so you take it. We follow on that same mindset that we want to spread the word, the cards are great but ultimately it is trying to, with that niche, finding what you want to do, we’ve had our best success then defining a vertical which is healthcare and then going after those people, right? And so, yes we still like I still hold the conference, I still do the podcast. Our health care people, in fact I don’t think if there is any health care people here?”, so we’re still doing things outside of healthcare before my marketing and like where we’re going to look for customers. We spent all of our focus. In fact, we’ve actually even gone as specific to customers of GE. And that’s who we’re targeting, trying to establish those relationships, increase our network width. We want to become the SQL Server people for GE centricity customers, and then once we do that we will go on to the next phase.

Steve: I think finding customers, that’s the hardest in the very beginning because you start out, you have no history there other than with your full time employer before that perhaps, and you’re trying to find customers. I think that getting the on boarding process for new customer the very first time always seems to take far more time than you ever expect because that first customer is everything to you, and without that first customer you have no income, you have nothing there. And then I found overtime once you’ve got that first customer, as long as you’re not breaking the rule of putting all of your eggs in one basket with just a single customer, the next customer is little bit easier because you have the work from the first customer to sort of subsidize the time while you’re waiting for that second customer to get things signed. And then once you’ve got more customers you end up getting repeat work. And I think there are some customers that you may work with that once you found them you’re getting work every single week for years to come, and there are others that once you found them and you’ve done work with them it may be you come back and work with them every six or eight months depending on what their needs are. I think that finding them, and I think a lot of it is really just the specializing. And I think in finding the customers you just find out and say, “Hey, I’m a computer tech guy. Hire me.” That is I think the wrong approach there because then you’re competing with everybody that’s a computer tech person anywhere. But if you specialize, like my specialization has been in database corruption. There is not a lot of people to compete within that area so that’s an area that people find me now when they come across corruption. And because of that specialty I’m able to take it on and fix them and get happy customers really quick right there or not always real quick, but generally as quick as we can. The other areas like the database health assessment. A lot of that has come out of Database Health Monitor. I think that just getting your name out there, whether it’s like what we do with the podcasts, or what we’ve done with the database health monitor or blogging. I mean just getting out there, social media, LinkedIn. Things like that to get your name out as someone who very rarely when you’re doing those kind things turn into an immediate like you do something and within 24 hours you have work. But it turns into you become that trusted name and then 6 months or two years later somebody remembers, “Hey, that’s the person who specializes in that. Let’s call him.”

Carlos: It’s definitely an investment, right? You have to wait a long time for those returns sometimes. It can be very very difficult. Ok. Well, thanks again Aaron for the question. I thought some interesting insights and conversation if you’re interested in talking a bit about this more. Maybe have some of questions, of course we much being at the summit or you can reach out to us via any of our social media channels and we’ll be happy to have a conversation with you. Give some thoughts around that.

Steve: So one of the things that I wanted to do was call out an interesting comment that I like from Randolph where he was describing us at SQL Data Partners and he says, “One is a bit odd, then there’s Carlos.” Thanks Randolph, I love it.

Carlos: Special thanks to Randolph and Jonathan for providing some insights there as well. Our music for SQL Server in the News is by Mansardian News under Creative Comments. If you have other suggestions for topics or suggestions about things we should be talking about, you can reach out to us on social media. You can find me at LinkedIn, I’m Carlos L Chacon.

Steve: Or you can find me on LinkedIn as Steve Stedman, and we’ll see you on the SQL Trail.

Episode 113: Standing on the shoulders of giants

“If I have seen further, it is by standing on the shoulders of giants.”  Those words are attributed to Issac Newton and they are fitting for the way I was feeling recently as I thought back to those who have helped me in my career.  While we might always want things to be better, as technologies we have it pretty good.  There are unnumbered people who have dedicated untold amounts of time so we can have the tech available to us.  We take a moment to think about those who have paved the way for us to be where we are now.

Whose shoulders are you standing on?  Let us know in the comments below.

 Episode Quote

“This idea that the way he was able to look at the world is based on all the previous work that has been done”

“I guess “gratitude” is the right word of those who have gone before us and enable us to have those who are in technology we have a pretty nice career.”

“There are just a lot of people out there who just want to help and just want to better things with what they do every day.”

Listen to Learn

00:38 Companero shout out to David Stoke
02:11 SQL Server in the News – SQL Server 2017 is out
02:35 Interleave execution for multi-statement table value functions
05:34 Artificial intelligence, analytics, machine learning
07:58 Today’s Topic: Standing on the Shoulders of Giants – having that gratitude to the people who have gone before us
11:27 Carlos shares a documentary he watched about bitcoin
14:50 Stop complaining
19:56 Who are “giants” in the life of Carlos?

Transcription: How do you start consulting?

Carlos: Companeros, welcome to Episode 113. It’s good to have you on the SQL trail again.

Steve: Yes, it’s good to have everyone who’s listening. Good to see too, Carlos.

Carlos: Yes, always good to connect back with you, Steve. Today, we’re going to be talking about, kind of changing the subject up a little bit, and the idea is standing on the shoulders of giants. We’ll get it to more what we mean about that in a bit later but that’s the idea of our topic for today.

Steve: Ok, sounds good.

Carlos: So we do have a companero shout out. I want to give a shout out to David Stokes. Now as we record this we actually have not had the Companero Conference just yet. That will happen, actually as soon as I hang up I’m going head down there and have the conference. But I want to give a shout out to David. David is the leader of the Norfolk user group. He was actually the first person to buy a companero ticket and has been a great supporter of the podcast and of the conference. I want to give a shout out to him and thank him for what he’s done.

Steve: Yup, you know the interesting thing about David is that he is the very first user of Database Health Monitor that I have ever met in person.

Carlos: Oh ok, interesting

Steve: I met him at Charlotte, North Carolina for PASS Summit, was that 3 or 4 years ago?

Carlos: That’s right, 2013 I believe

Steve: 2013, ok, so 4 years ago. Yeah, it was really kind of cool to meet him and listen to what he had to say about Database Health Monitor back then and he’s been using it ever since. David is a friend of Database Health Monitor.

Carlos: Yes, very nice. Ok, as mentioned, I’m sure the conference went great. You already have that and we’re looking back to some feedback. We will probably have some kind of post mortem about that. But now I think it’s time for a little SQL Server in the News. We’ve been going back and forth a little bit about this and one of the things I wanted to talk with, of course we know that SQL Server 2017 is out and should be up, right? I’m sure you spent your weekend installing that. But one of the things that I wanted to discuss a little bit here which has been mentioned I think it’s expanded a little bit here and that is the interleaved execution for multi-statement table valued functions.

Steve: Oh yeah, this sounds pretty cool because there’s been so many concerns over the behavior of multi-statement table valued functions over time.

Carlos: That’s right. Yeah, so ultimately the bad news is that in previous versions of SQL Server, and I guess in 2017 as well. I mean, there is a fix for it but the optimizer can’t always determine how many rows the table valued function is going to affect. As a hard time just in for that fact, right, and so in previous versions basically it would guess a hundred and then it would really be 5,000 and so we have a bad plan, and that was kind of the issue. So what they are going to do here with 2017, they are adding some analytics and some machine learning basically into the optimizer, so that when the first time the plan comes in with the table valued function it’s going to guess again a hundred rows when it executes it’s going to have 5,000 rows. The difference now is that the second time it runs it’s going to take a peek at that previous plan and say, “Hey, how did I do?” And it says, “Oh, I did horrible. Let me adjust my numbers and take in the previous execution into account to see if I can come up with something better.”

Steve: You know, I wonder as I hear that if that is actually going to help or hurt because. I mean, if you table valued functions commonly have the same number of rows they are processing it might help you. But what happens if you’ve got such a wide variety of data in your database that every time that gets called the table valued function has a very different set of data that it’s processing depending on a client or customer or whatever it may be.

Carlos: Right. Yeah, and so obviously we have skewed data it makes it difficult to solve for the outliers. There still probably an outlier case. I think here what they are trying to get at basically is do I use a hash or do I use nested loops? Which way that I’m going to go about that? I think, again, after you execute it a hundred times you’re going to have some data there to then be able to say, “Ok, well, 80/20 rule. This is the way I’m going to go with.” I guess the implications here are not suggesting all of a sudden that everyone start using table valued functions and this is going to solve everyone’s problems there. I guess what was curious, or what was interesting to me is that I thought this was a very interesting way because we hear a lot about artificial intelligence, analytics, machine learning even. This is now a problem that we deal with all the time and now Microsoft is putting that analytics, that machine learning into the product that we love. And I thought, you know what, I meant that’s kind of a very specific scenario. I wonder if we couldn’t start taking some of the same thought and then applying it to other areas of our business so that our users or our customers might also benefit, so that was kind of my thinking there

Steve: Right, so it will really be interesting to see if somebody comes out with like an AI package for tuning SQL.

Carlos: Yeah, something like that, right. Or even that idea of, “Ok, I have this data. My customers keep asking me for whatever use cases. Can I apply some analytics here to give that better information? So it will be interesting to see how this continues to evolve. I think this idea is going to affect those of us who continue to be administrators or gatekeepers of data. These are going to be problems that we’re going to have to start solving, so it will be interesting to see how other tools crop up and try to address other problems.

Steve: Yup. Yeah, and I think it’s been easier with just sort of fix rule sets previously to be able to definitively say this is how something behaves. But with AI built in on how those things are being handled and how it’s tuning itself almost, it may be harder or maybe more work to figure out exactly what’s going on when there is a problem

Carlos: Sure, and I think that’s goes back to you are still going to have to know your system for those outlier cases. What’s going on? Why are you doing what you’re doing and is there a different way to potentially go about it?

Steve: Yup, interesting.

Carlos: Ok, so today’s episode, the show notes are available at sqldatapartners.com/giant

Steve: Or at sqldatapartners.com/113 for our episode number.

Carlos: Yeah, so ultimately this idea, and maybe a became a bit nostalgic which we can get into in a second but Sir Isaac Newton is quoted in 1675 as saying, “If I have seen further it is by standing on the shoulders of giants.” And ultimately this idea that the way he was able to look at the world is based on all the previous work that has been done, right? Helping him to arrive to where he is now. I got a bit, as I mentioned, nostalgic about this idea where I ran into to somebody who work with a former co-worker of mine. We started talking, and so this former co-worker is actually the person who convinced me to apply for my first database job.

Steve: Oh, I remember that story from one of our previous podcast.

Carlos: Yes, and I wasn’t actually completely qualified for it. But he convinced me and he coached me and said, “Hey, I think you do great.” And I think back to that, obviously, you know, there are thousand different I could have gone but I think about where I am now and ultimately kind of that first step of making that decision. And so I think about it, I guess “gratitude” is the right word of those who have gone before us and enable us to have those who are in technology we have a pretty nice career. Yes, it’s not all roses and sunshine all the time but the fact that those giants have gone before us, many of whom we’ll probably never know. We have to look up in Wikipedia, and the fact, even the first people who put together the rules for the transactional database. I know he was brought up in a keynote, unfortunately I can’t remember his name, obviously on top of my head. But, I mean, they put in place what we have today. And I think it’s easy, and not to discount. I don’t mean to discount them at all but it’s easy to look at the Bill Gates, and the Steve Jobs, and the, what is it? Oh gosh, Linux is going to kill me now

Steve: Linus Trovalds.

Carlos: There you go, thank you.

Steve: I don’t know if I pronounced that right

Carlos: If not, I’m sure you’ll get corrected. Again, not to discount their contributions but they are not the only ones. You think about all the teams behind them that helped them do what they wanted to do.

Steve: Yeah, I guess with that, I mean you look at a product like SQL Server, it’s really been like a Microsoft product since 1989/1990 but it was being worked on for years before that and then before anybody even started working on what became SQL Server, there were people in the 70’s that were theorizing on how these databases would work and the rules around it. I mean, 50 years worth of work, and foundation that has gone into what we have today in SQL Server 2017

Carlos: Right, and how amazing it is. It was interesting to kind of [term unclear – 11:26] here slightly. I was watching a documentary on bitcoin, and so bitcoin is the digital currency that they are trying to advance and I didn’t realize but, oh gosh, this Swedish guy who started Wikileaks. Whatever his name was, he was part of the original group of this eight people who put together sort of theorizing about digital currency, and they came up with a block chain idea and methodology. It was because of his situation and what Wikileaks was doing that really kind of put bitcoin on the map because when PayPal took their services away and said, “Hey we’re not going to let you use PayPal services.” That’s when he was like, “Ok, well, bitcoin, Internet can you help me,” and kind of launch that whole idea. In a way it was very interesting. But anyway, in the documentary they go and they talk about and they say, “The person through the door with a new idea get shot.” Anytime you have something revolutionary, the first person through get shot but somebody has to go through the door. So they talked about some of these first people like the first companies that started in New York City trying to exchange currency. For example, dollars to bitcoin and things like that. Well, he’s going to jail. Long story, but you can watch the documentary. Anyway, but I thought it was interesting. I guess going back to this idea that all of the ground work that was put in to enable us to enjoy what it is that we enjoyed today.

Steve: Right. But what was interesting, I mean, he was going to jail for Wikileaks side of things not for bitcoin side of it.

Carlos: Yeah, I’m sorry, so Julian Assange. That’s his name, the Wikileaks guy, and he is going to jail for something else. I don’t know it. But the guy in New York City who was doing the bitcoin exchange, so he was actually trying to get people on the bitcoin and basically sell them, bitcoins this idea. He is going to jail because one of the guys, unfortunately, illegal activity tends to push a lot of innovation. And so when you have something that is not monitored there was some illegal activity going on. He knew about it, and he was basically selling bitcoin to people who were doing illegal things and so that’s why he’s going into jail.

Steve: Oh, interesting. Ok

Carlos: It wasn’t that he was doing anything illegal but he was kind of aiding, and abetting if you will in a sense.

Steve: Wow! I hope I’m not considered aiding and abetting by investing in bitcoin. Or maybe the word is “gambling” not “investing”

Carlos: Yeah, that’s right. It is still a bit shaky, again to progress for a minute, the value of how that gone up it was pretty impressive but obviously it has come down I think in 2011 or 2013 something like that. And so where I was going a bit with this as well is this idea of it kind of grits on my nerves a bit, and we all like to complain. We all find ourselves in situations we don’t want to be. When people will say, “Gosh, I they had only done this right. Why they have to do it this way?” Basically, I’m inconvenience and I don’t really appreciate all the things that I have

Steve: Right, and sometimes that inconvenience comes from just not understanding what the people did before you and just complaining about it. I mean, I’ve done that but I’ve seen that happen plenty of times.

Carlos: Yeah, that’s right. You have that “Aha” moment where you’re like, “Oh, so that’s how it works.” Sure, you may have developed it differently; you may have architected it differently all of that jazz, great. But, you know, you work there, right? And now you’re now kind of taking advantage or taking over in some case. No I’m not saying it’s all always great but the decisions that were made there before you have you put you in a position where you can now either take over or ride that bus if you will. Yeah, it’s kind of interesting.

Steve: Yup, and you may look at something and say, “Wow, that’s stupid. Why would they have ever done that?” But then you look back 10 years ago at what was being done, or 15 years ago, and that was cutting edge technology and the fact that it even worked at all was completely brilliant.

Carlos: Exactly, exactly. Yeah, you know it is funny particularly as database people, like yourself Steve, there are developers among us or recovering developers or whatever you want to call them. But we like to complain quite a bit about you know, and the stupid developer if you’d only done that. Well, I can kind of find that we’re being a bit of a hypocrite because we use software all the time, on our phones and on our computers. I mean, we wouldn’t be able to do anything that we do now if it weren’t for software. We need to be mindful of where we are, what has been invested and give people a little bit of break when they are not upgrading to the latest thing. I guess in our last episode even was about patching. You know, just staying up and keeping with the latest and can kind of get a little tiring sometimes, and so you’ll forgive people a little bit if they are not quite there to where you are.

Steve: I think some of that comes back to what I learned in a class. It was called the Dale Carnegie class that I took several years ago. It was based on “How to Win Friends and Influence People” book as well as some other books, and it was a simple statement that said, “Don’t criticize, condemn, or complain.” It’s just that one of those things that just comes up and if I find myself criticizing, condemning or complaining, I try and stop myself. I try, so it doesn’t always happen but I try.

Carlos: Yeah, it’s one of those things, the reality in today’s world with social media everybody has a voice box or a [term unclear – 18:11] and they want to get up. It’s not to say that reform is not needed, that we can’t make progress. There are times when change needs to happen. There’s no question on my mind. I think it is just more about the way that we go about it and then recognizing what we have and how do we go from there.

Steve: Yup, and you know, no matter how bad the day may be we’ve got things pretty good right now. I think that’s so often overlooked in, well you mentioned social media, I mean, if you’re on social media and it’s almost like everyone is out there just to complain

Carlos: They are either having the best day of their life at the beach with the vacation photos or the world is against me. I think those for you companeros, those who are listening, going to be in technology, yes, you may not be where you want to be. And I’m not saying that you should settle. That’s not what we are saying either but I do thing that you if you take a minute to use the phrase from our culture if you will, is to capture many blessings and just to kind of appreciate where you are and know that there is a path ahead. If you will continue to work there, like we talk about in the beginning, all these artificial intelligence and analytics; yes, is it going to affect you, is it going to change what you do? Absolutely, but hopefully that will be to your advantage and not to your decrement.

Steve: Right, yup

Carlos: Anyway, we thought we change us up just a little bit and share some of our thoughts on that topic. I guess we’d like to, again, thanks to all those shoulders we stand on, so Robert Pollard, he was the guy who commits me to get to that job interview. I even think about Matan Yungman and Guy Glanster. The other guy is the SQL Server Radio. They are out there doing the podcast. Listening to them made me think, yeah, let me try this. Right, let me do this as well. And then of course all those in the community who have give freely of their time. I guess especially thinking about that to all of the speakers who came to the Companero Conference. They put a lot of time and energy and I’m super appreciative to them and what they’ve given.

Steve: And you know, it’s just interesting because one of the reasons that I enjoy what I do so much I think a big part of that is because of the SQL Community. You don’t see, at least I have never seen a tech community similar to what we have with the SQL Community. There are just a lot of people out there who just want to help and just want to better things with what they do every day. And I think that’s awesome

Carlos: Yeah, that’s one of those things. I think standing on the shoulders, again, I’m sure there will be a lot of people out but I kind of, if there’s one person I would point to, Steve Jones. So Steve Jones, probably one of the most humble guys you’ll ever know. He has done quite a bit in the community, kind of a very public figure but he is always looking to help and do so in a way that he’s not boastful. He is just trying to help you along and he has taken his breathe of experience and freely gives of that and all things to do this.

Steve: Yup, and what’s interesting with Steve Jones is that when he presents and gives back and shares with people. He does a great job in the tech side but he also does a great job on like the personal improvement side. Like, how do you get out there and improve the position you’re in – how do you improve your social media footprint, how do you improve your value, all those kind of things. I’ve seen him present a few times and definitely another giant to call out there.

Carlos: Yeah, it was funny, so we have him on the podcast that was one of those first ten episodes and looking to expand your game. And one of the things that I really liked about Steve, because again, he is one of these internet famous folks and he is also one of the few of that genre that will come up to you and say, “Hey, I’m Steve, what’s your name?” You’re like, “Ahhh. I forgot, I know who you are.” Of course he is in a Hawaiian shirt. He has a great brand but part of that brand I feel like is his contribution, and because of that he has made it the norm or almost the expectation of this is what the community does.

Steve: Right, yup.

Carlos: So awesome, so that’s our episode for today. Companeros, thanks as always for tuning in. Our music for SQL Server in the News is by Mansardian used under creative comments. If there is other topics you would like for us to talk about or ideas, you can hit us up on social media. I am carloslchacon.

Steve: You can connect with me on LinkedIn stevestedman, and we’ll see you on the SQL trail.

Episode 112: Keeping up with patching

When listener Mel Vargas first suggested the topic of patching, I was not sure this would work as a topic; however, the Equifax story had just developed and there are many other security related issues that could be prevented with patching.  Match that with our guest, Robert Davis, who just happened to publish some articles on patching and I decided we needed to do this episode.

I should never have doubted this as topic.  While we are a bit more protected in SQL Server than others, the security threat is still real and this is something everyone has to go through.  Robert presents us with some interesting details on how he goes about patching and we think you will find the episode compelling.

Episode Quotes

“You not being able to be patched for the new stuff, you are really opening yourself up to a variety of attacks.”

“We’re all on the same team ultimately and the success of the application really benefits all of us.”

“There isn’t a security reason to be diligent about patching SSMS.”

“The really good DBA, really proactive one, gets overlooked because everybody else in the company doesn’t see the fires that you’re preventing.”

“Learning to speak up for myself and to publicize the things I do definitely was one of the best things I had to learn how to do.”

Listen to Learn

01:08   Episode Topic: Patching
02:19   Why should we care about patching and what it helps to do?
06:26   Maintenance window, automated patching and system center
07:56   Octopus: Patching Automation Tool, automation tools
2:52   Deploying and rebooting, and restarting services
15:48   Do you really need to update SQL Server Management Studio for security reason?
19:44   Cumulative updates in Azure
22:34   Windows patches and SQL Server patches, failover cluster, availability groups
26:39   Patching testing
29:07   Scenarios and issues when trying to do install updates or patching
36:32   SQL Family questions

About Robert Davis

Robert is a SQL Server Certified Master, MVP, and has spent 17+ years honing his skills in security, performance tuning, SQL development, high availability, and disaster recovery. He served as PM for the SQL Server Certified Master Program at Microsoft Learning, and in various roles at Microsoft specializing in SQL Server administration, development, and architecture. He currently works as a Database Engineer at BlueMountain Capital Management where he spends a vast majority of his time tuning massively parallel queries. Robert feeds his passion for security by acting as co-leader of the PASS Security Virtual Chapter.

Episode 112: Keeping up with patching

Carlos: This is Carlos Chacon.

Steve: I am Steve Stedman.

Robert: I am Robert Davis.

Carlos: Robert, welcome to the program.

Robert: Thank you.

Carlos: Yeah, it’s good to have you. Thanks for taking a little time of your busy schedule all the way from New York City to come and chat with us today, so thanks for stopping by.

Robert: Thanks for having me, coming at you from the heart of Manhattan.

Carlos: Wow. Yes, deep downtown.

Steve: Well, I know over the years on different topics that I’ve googled on around SQL Server often times some article or post that you’ve written has come up and definitely appreciate that, it’s always been good content. I want to say thank you.

Robert: Wow, thank you for that feedback.

Carlos: Yeah, and it’s nice to have you on the program today. Ultimately our topic is going to revolve around patching. Maybe not the most sexy of topics but as administrators something that we all have to live with, and I think has only accelerated with the “new” Microsoft and not only we’re getting patches now, we’re actually getting versions and all these things. And so keeping up with all of these can be laborious at times. And so we wanted to start this conversation with talking a little bit about patching. And maybe let’s just start with the basics, right? You know, there’s an inevitable or at least in the past it has been I’m going to wait to upgrade until I get SP1, right? And then, we’ll go forward with the upgrade. And so I think a lot of people look at patching for lots of different reasons. I guess let’s just go ahead and hit on a couple of ideas of why should care about patching and what it helps to do.

Robert: I’d like to just point out the fact that we had a couple of major events very recently in the computing world where there were vast instances of people’s servers being encrypted and it was utilizing very very old vulnerabilities that have been patched years ago that the people affected by them were the people that weren’t patching their servers.

Carlos: Or even learning older versions of older operating systems and things like that and then just have them patched.

Robert: Right, exactly. Those were two major events that could have been stopped right away if people just have their systems patched up to date.

Steve: Interesting. So I mean I have come across situations where people have said, “Oh we’ve got SQL Server 2005”, running on whatever older version of Windows Server it is that they are happen to be running on, “It does everything we need for our business and we don’t need to spend money on patching or updating it because it’s getting the job done for us.” What would you day to someone with the comment like that.

Robert: I would say to those people that if for these really old versions that are no longer supported that means you’re not getting patches for these SQL instances anymore. When you see things like these encryption attacks or SQL-Injection attacks and a lot of these know vulnerabilities attacking people’s systems. In fact, just recently we had a reoccurrence of a mass Slammer attack. If you remember, Slammer hit SQL Server 2000 pretty hard, and suddenly it made a comeback earlier this year. And obviously the people that are going to be affected by that are the people running SQL 2000 that aren’t patched and aren’t getting patches anymore. So by not being on the current version and you not being able to be patched for the new stuff, you are really opening yourself up to a variety of attacks.

Steve: Yup.

Carlos: And so then this kind of gets in to the heart of I think some of our conversations so some of those vulnerabilities. So the worms that were introduced earlier this year there was the patches available 30 days prior. So if you would patch in those 30 days it would have been fine. Some were much older, then we get into that idea of how often or how frequently should we applying them and the testing that goes around with patching? And again I say this is not, I don’t know if anybody that really enjoys patching. I don’t think I’ve met a person that person, right? So I guess thoughts on how can we introduce a process or a culture even to get those patches in and get them tested and maybe what’s necessary to kick the tires, because we know that you as much as we try a lot of times our lower environments aren’t quite the same as our higher environments and that’s a problem for a lot of people.

Robert: Yeah, exactly. But I think really the key to getting people on all our kaitens of regular patching is to introduce automation to the process. Obviously if you’re up late at night installing patches manually waiting for them to finish and then rebooting is going to be a boring tedious task and nobody is going to want to do it. We use system center to push patches out to all of our servers, and with system center we have the ability to. If we have some exception like we don’t want it to be automatically patched and rebooted in the middle of the night we can exclude certain server so we can manually do some certain server but otherwise at a certain point our automation kicks in says, these servers aren’t patched, patch and then reboot them starting at 11PM on Saturday night.

Carlos: There you go, so kind of finding that maintenance window and then making that happen. So you mentioned the system center, right and obviously if you have that product and I’m sure it’s doing a lot more than patching there is some inventory kind of helping you with your server environments just in general. Have you played around with any of the other patching automation tools by chance?

Robert: Yeah, actually when I worked at Microsoft we used Octopus for our patching processes. We would have Octopus, it would push the patch files out to each of the servers and install them and do anything. Octopus is great because if you need to do anything like stop a service before you patch and then restart it after you patch, you can build that right into the Octopus script. So for example, say if we needed to stop a web service that’s getting the web, the database server before we patched and rebooted it we could do that into the Octopus deployment script. It would stop the web services, install the patches and reboot it and then restart the web services on the web servers. It could do the whole, anything that could be automated that could do and the really great thing about Octopus too is we could build into it. Some sort of some basic testing at the end of the deployment like it could go in and send a request to the web service and check to see if it gets a valid response back so that we know. Yeah, we know that the web service. We would setup like some dummy process on the web server that Octopus could hit it would make a database call and it will just return a value, and if it returns the expected value then we know that the database and the web servers are all functioning properly.

Steve: Very interesting. I mean, I have worked in environments were Octopus deploys are used but I’ve never really thought about using it for updates. Kind of makes me want to go back and take a look at it now.
Carlos: Right, well I think it’s a good point too that Robert makes in the sense that you want to automate those but then with automation comes with responsibility in the sense of then going and testing some of those things. I guess there’s a little bit of that fear particularly if we’re going to start restarting services. You want to make sure everything comes up, and so automating the patch almost like it’s step one, right? And then there’s building out the test afterwards to complete the process.

Steve: And that’s the interesting thing being on the DBA side of the world. Oftentimes you don’t always understand that entire application or web service or whatever it may be that’s using the database and know how to even tell if it’s working correctly. And I think that if you can have the tools that automated it that would really be great.

Robert: Absolutely and it definitely wasn’t something that we had done on our own. We had some amazing test people and developers for the application I worked on at Microsoft and they were really proactive about helping us build things like this and creating the web service call that we’ve made then do a dummy transaction in the database and make sure everything was working.

Carlos: It takes a village to do automated patching.

Robert: Absolutely. It often gets lost but we have to remember Dev Team, DBA Team, whoever we are we’re all on the same team ultimately and the success of the application really benefits all of us.

Carlos: Sure, it’s almost taking me back to the conversation that we have with Richard, Steve, about building trust our teams. This is kind of an interesting scenario where it gets to be a win win. One because we get to apply the patches that we want for security purposes but then application people can be assured that the application is still available when we’re done with it.

Steve: Right. And if you have that all automated then it’s less extra hours in the middle of the night for all of those people involved as well.

Carlos: Exactly, which I think everyone would agree is a good thing. I wanted to ask about deploying as well and restarting, you mentioned restarting the services. Is that across the board? I think with SQL Server we tend to it a little bit more just because SQL Server is running, we’re applying that so there’s file dependencies that sometimes get lock and things. Kind of by rule are you restarting your services after you patch them?

Robert: Typically, it’s easy to determine whether or not the automation process you can configure it to reboot it if reboot is recommended. You know, like some patches will say you must reboot to complete installing this patch. Other patches like say applying a CU to SQL Server won’t require you to reboot but then applying the new version of SSMS probably will require you to reboot. Most of the automation or the system center patching that we use gets that message back as to whether or not reboot as needed and then we’ll reboot as needed. You can also override that if you have systems that you don’t want to be rebooted automatically. For example, if you have a system that has a known issue where like maybe when it reboots all the drives don’t always come back online. I have seen cases like that where until they got configure out why some of the drives weren’t coming online they had configure it to not reboot automatically. But in the past without having that when we schedule for patching we to take into account that reboot maybe required and so we schedule Maintenance Window according to how long we think it will take if a reboot is going to be required. And so if you got the time, in the past we just install it automatically whether it says to be rebooted or not, reboot it. And there have been few updates from SQL Server, few CUs, or service packs where it gave me no message saying it needed to be rebooted, so we didn’t reboot. And then the next time I go in there to try and install an update, I then later go in to try and install the SSMS update and it says, can’t install there is a reboot pending. Like, why didn’t you tell me? So if you got the time scheduled and you don’t and you just want to play it safe. I’d say, yeah go ahead and schedule the reboot if you don’t foresee it causing you any problems.

Steve: Alright, so one of the things you mentioned there was the patching of SSMS or SQL Server Management Studio and one of the things that’s interesting around that is that, I mean it’s used to be the Management Studio was part of the SQL server install and if you updated SQL Server you would get your Management Studio patches at the same time. But now Management Studio is detached, running separately and with a completely different patch cycle and we’re getting much more frequent updates there but the other thing is it’s not a very smooth update process like a lot of the Microsoft products desktop applications for instance. You will get a message that says, there is an update required or come part of the Windows update process. But with Management Studio pretty much you got to always download it and then install that update. I’ve talked to a lot of people who, I don’t know, they kind of a little bit bothered at how annoying Management Studio is with monthly updates. And I’m just curious what your take is on updating Management Studio? Because some people I’ve talked to they come across to say, “Well, I’m not going to update Management Studio every month because I don’t need those new features.” But do people really need to update management studio often for security purposes?

Robert: With SSMS there’s really not a big security issue with the SSMS. Nobody is going to take away SSMS on your machine and be able to do bad things with it. Like they could took over your SQL Server instance. So there isn’t a security reason to be diligent about patching SSMS. But they are trying to work out a lot with the process and a lot of the recent versions of SSMS have had bugs in them. And there maybe things you don’t notice right away for specific features that you rarely use but you can definitely look at the update to see what they’ve done and determine if it’s something you need to do. But for myself every time an SSMS update comes up I update the SSMS I have on my desktop. I rarely ever use SSMS on the servers themselves. I like to have SSMS on the server in case there’s a critical issue and the only way that you can connect is via the dedicated admin connection. So I would like to have the SSMS on the server but I almost never use it so I don’t update the SSMS every time on the servers, every time an update for it comes up. But what I will do is when a new CU or SP comes out then I’m going to apply it to my instances. I’ll get the latest version of SSMS and I’ll install that as the same time as I do the CU and updates. I know that SSMS is on the servers themselves is at least as current as the last patch that was applied for the SQL Server itself.

Steve: Yup. Ok, great.

Carlos: Ok, that’s an interesting approach.

Steve: So I knew you mentioned accumulative updates there as well. It used to be like maybe 8 or 10 years ago that cumulative updates weren’t really as safe as they are today it seemed like, and that people would generally just stick with service packs when they are doing patching unless there was a specific reason that they needed cumulative update. But it seems like today cumulative updates are strongly recommended compared to where they were in the past. I’m just curious on what your feelings are on that or your experiences.

Robert: There’s used to be a big topic of debate amongst DBA’s whether or not to install CUs. A lot of people said, they always install the latest CUs. I was one of those that always said, “I only install a CU if it has something that specifically fixes an issue I’m having.” For the very reason that they didn’t get CUs used to not get full regression testing. When they were working on 2016 one of the things they change is that CUs not get the same full regression testing that service packs get.

Carlos: There you go, that’s the change right?

Robert: Yeah, so for me that really took that element of risk out of it. So now I will install a CU at my first opportunity to perform CU updates on the instances.

Carlos: I think that’s the effect that Azures had on the environment and that they need to be able to test those to support the Azure environments.

Robert: Absolutely. And everybody who owns Azure gets those updates automatically already so it was really one of the reasons say had the building the same level of trust for their CUs as they do for service packs because if you’re in Azure whether they’ve taken that choice out of your hand you’re getting the updates whether you want them or not. It’s really one of the selling points of Azures is that you don’t have to worry about the updates. It’s all taken care of for you and it happens automatically without you having to do anything.

Carlos: Right, also true. So before we go on with the other scenarios because I had a lot of questions on, so are you bundling your Windows patches and your SQL Server patches or do you like to do those separately?

Robert: We’re not bundling them, in that sense they are separate.

Carlos: Bundles is the wrong word. I guess what I mean to say, ok it’s patching weekend I want to do all my Windows updates and my SQL Server updates at the same time.

Robert: Yeah, so that’s really the kaitens we’ve gotten into now, and all of our production databases are on availability groups. The way we do it is we always patch the inactive nodes one weekend and do all the patching of the inactive nodes and then we failover our availability groups so that the patch instances is now the currently active node. And then we will run with that for a couple of weeks and then we’ll do the other CU. Unless there are some issue that we feel we need to update sooner we’ll do that another weekend a couple of weeks later and then throwback again.

Carlos: Ok, and then back into this next architecture if you will. I mean, so many people, they are stand alone where there’s clusters, now availability groups. And it sounds like if you’re familiar with the clustering idea that same concept, I can apply my service pack to one, flip it over, apply it to the other, and then either to flip it back or whatever your policy is there. That hasn’t change much with the availability groups.

Robert: Exactly. I mean with the availability groups you still want to patch your inactive node just like you would a failover cluster, failover and then do the nodes that are left that used to be active and then now inactive.

Steve: But it also sounds like either way with that whether it’s an availability group or a failover cluster that you’re delaying the time between when you patch one half of it versus when you come back and do the rest of them, so that you can have time to know if something went wrong and the ability to flip back to the old one if something did go wrong.

Robert: Absolutely, and because the way we do it where we break it up with the nodes to different weekends. If we do run into an issue like if there is a breaking change in the new version which is a lot likely to happen with the service pack than with the CU, but if there is a breaking change and the secondary can’t continue syncing data because there are some upgrade in the database that it’s trying to do but it can’t because it’s the secondary. The type of behavior you would see if you had a major mismatch between the nodes, you then have the option because the unpatch side is now the inactive one. Typically, you don’t even need to wait till the middle of the night. If you say, “Ok I’ve got to get this patch right now.” You’re not going to cause a major outage by now going in patching the secondary to make it match the active database.

Carlos: Yes, then you’re back in sync and hopefully that fixes the issue and they can continue to replicate.

Robert: Absolutely.

Carlos: Yeah, very cool. And then testing, right? We talk about automated deployments, right? Any specific test that you do around admittedly and I guess you know, maybe I shouldn’t admit this, but I think a lot of times my testing has been, “Oh, I install in a lower environments.” The proverbial, wait for something to blow up, and then if I don’t hear anything I move forward, right? Thoughts around patching testing?

Robert: That’s definitely one of the things I’ve always done is I start with dev and then move on to test and then move on to production. So like when the new CU or SP comes out I’ll install it on our dev machines the same day that we get it. And that gives us some time to see if any problems arise there before we go there. Also, usually if there’s a major problem with the CU or SP usually you hear about it within two or three days of the patch going public. Because a lot of people are start installing it on the devs environments right away and if there are problems they start finding it almost immediately. By the time we get around to patching the first set of production servers not only I’ve had it running on dev and test for awhile. A lot of other people have two and a lot of these people are very vocal when they find problems and really put it out on social media, which is one of the things I love about things like Twitter because you really, and some may post out, “Oh, this new CU is breaking my machines don’t install it.” Even if it’s somebody you don’t follow people will start retweeting it and you’ll see it pretty quickly on social media.

Steve: Yeah, that’s interesting because that really shows how valuable social media can be in that case where, I mean a lot of people look at social media in the workplace as a time snake in complete ways but when you’re using it for things like that it’s incredibly valuable. Ok, so one of the things I really want to ask about and it sounds you’ve done a lot of patching, do you have any horror stories either that you’ve seen happen or you’ve heard of happening where somebody tried to do a service pack or an update or a new version of SQL Server and something just went horribly wrong.

Robert: I have seen a few of those. There were some well known bugs that got hit in CUs and service packs where the upgrade would fail, the first part of the upgrade would fail but then not always but sometimes if there are system updates that have to be upgraded. You know, it will run a script to upgrade those objects and it puts the database in a state where if at that point it fails it puts it in a state where it won’t come unwind because the database is marked as in an upgrade. So if the CU or SP fails at that point it will uninstall the upgrade but it doesn’t revert that database and take it out of that upgrading state. It requires manual intervention. Obviously, one of the easiest ways to fix it is to, and we only know this because we’ve had to go through these issues where the servers won’t start.

Carlos: Life experience here.

Robert: Yeah, find the script, try to run, start the server with only master running and run the upgrade script and let it complete and maybe it will come late when you run it manually, if not it may give you a clear error, pin you exactly where it was. Last time I ran into this is because they made some changes to the job systems, the way they track jobs in MSDB. And part of their script was it deleted all of system categories by default and then inserted a bunch of new system categories, and had the IDs for those categories hard coded expecting that the likelihood that somebody has created a custom ID or custom category and that it’s using one of these IDs were going to use. They basically just assume that those category IDs wouldn’t be taken up, so we had custom categories that we had created and those IDs were in use and so the script failed because there is a unique constrain on the column. These were really kind of basic things we would yell at our developers if they didn’t check for if they’re going to insert hard coded values and not check to see if they exist already. But those get reported, they fixed those issues, and we haven’t seen that kind of thing…

Carlos: Click, yeah, record again. I apologize, Robert. If you’ll just go ahead and just pick up that story again talking about the change they made to the schedules or the categories then we can keep going.

Robert: Yes, so last time I encountered this issue myself was because they were upgrading the job system in MSDB which involved, their script would delete all the system categories from the categories table and then reinsert a bunch of new categories. The new categories was actually about as twice as many categories and they just made the assumption that the IDs, the hard coded IDs, they were inserting for the categories wouldn’t have been taken up by any user created category IDs. But we had created several categories of our own so when the script ran it had the unique constrain error because some of the IDs they were trying to insert were already in the table, and so it failed in the middle of the upgrade. So I found the script that they ran, ran it manually, got the error that it was generating. And I say, ok, so I went there. I dumped out the data from the table into something so I could fix it later, and then just delete, emptied out the table, and rerun the script and it completed successfully. These issues have all been reported to the team. I particularly gave them a hard time over that one because that’s the kind of stuff we would yell at our developers if they did. Like, why are you inserting hard coded values in not checking to see if they exist already?

Carlos: Or nothing else at least go to like 500 or something, right?

Robert: Right. But they took all that information in and they address those problems and these things never happened for a really long time. The only recent occurrences I’ve heard of these issues happen for people that are installing old updates. Like somebody is installing, “Oh look I just found out there’s Service Pack 3 for SQL 2005. Let me install that”, and opps it fails.

Steve: Interesting as you say that. I mean can we even get service packs for 2005 anymore? I remember recently somebody looking for one and we couldn’t find it anywhere on the Microsoft side.

Robert: Right, so you can’t get it through self service, you can’t find it yourself, but if you need to you can call my reserve support and they can get you old service packs.

Steve: Ok.

Robert: They are just going to ask you why you don’t upgrade. Obviously they are going to want to know why you don’t just upgrade to supported version. But they only keep the current versions of the updates online for you to download, so if you need really an old one you’re going to have to call support or I sometimes see people posting it for money. “Service Pack 3 for 2005, does somebody have a copy?” I see a lot of people are exchanging old updates that way.

Steve: Ok, interesting.

Carlos: Well, awesome, very good conversation, Robert. We do appreciate it and some good food for thought there. Shall we go ahead and do SQL Family?

Robert: Sounds good to me.

Steve: Ok, so Robert, how did you first get started with SQL Server?
Robert: You know, I’ve been in love with computers for as long as I can remember. But I didn’t go into IT straight away when I got out in the real world. I actually worked with the [term unclear – 36:58] disabled people for the first 10 years of my, that was my first career for 10 years. I worked with the [term unclear – 37:06] disabled people. But I was really longing for something different and still have that passion about computers. I’d started doing freelancing work, was doing Perl scripting and other types of web development. And eventually I decided, ok, it’s time to make the jump into the IT field for real. And so I started looking for position as a web developer. That’s the cool job that everybody wants though, right? Everybody come out of college wants to be a web developer. Competition was very stiff for web developers. I went to several interviews and ultimately I got offered a position as a database developer. And I thought, ok, I’ll do this for awhile and try to work make my way into web development world there. I worked for that company for 6 years and over the years, they were a very small company and their needs change and sometimes I would do web development. Sometimes I would do application developments, sometimes I would do database development. But we came to a point three years into my tenure there where our DBA had left the company 6 months earlier and they had never filled his role. And they were telling me that they thought they were at a point where they could make me a web developer full time. And at that point I realized I’d no longer want to be a web developer. I really enjoyed working with the databases. So I made a counter proposal, told them, at that time I was the head of the database development team. So I proposed that I take on the role of a DBA and still continue being the head of the database development team instead of doing web development full time. So that’s how I got my first DBA position. So that’s how I really got into it.

Carlos: Yeah, funny how things change, right, over time.

Steve: That point of enlightenment when you realized you don’t want to be the web developer. I can remember a similar point in my career as well. It’s an interesting time.

Robert: I like to tell people I’m a recovering web developer.

Steve: Oh, I like that.

Carlos: Now, of all the experience you’ve had in SQL Server and SQL Server is changing all the time. But if there’s one thing you could change about SQL Server what would it be?

Robert: What I would really like to do is there is some really horrendous features in there that I would love to see go away.
Carlos: Just take them out and shot.
Robert: Right. But that’s not likely to happen. If I had to narrow it down to just one thing, I would say get rid of all the features that have been deprecated for years.

Carlos: Pull the plug already.

Robert: I mean as much as I love database mirroring. I mean I wrote a book on database mirroring. Get rid of it. It has been deprecated for much longer than things are supposed to be deprecated. I mean there’s an extremely long list of things that have been deprecated since 2008, 2012. It is like get rid of these things already. We have people that are still using them and the logic they use is, they said that were going to get rid of it after two versions and they still haven’t, so it’s obvious that they are never going to. Why should I bother switching to the new stuff? People are still using the SQL 2000 system tables instead of the current catalog.

Steve: Right, right, so force deprecation. I like it. So what is the best piece of career advice that you’ve ever received?

Robert: I would say the best piece of career advice I ever received was that you have to be your own advocate. Early in my career this was something I really had a hard time with. I’ve never been one to go out and say, “Look what I did. Isn’t this great?” But one of the hard things of being a DBA is if you’re really good at your job nobody notices. People only notice the DBAs who are always fighting fires. If things are always breaking and you’re constantly in reactive mode trying to fix, people are like, “Oh, he is a great DBA because he is always fixing out problems.” And then there’s the guy who’s, his servers are not experiencing these problems because he is proactive and he prevents things from happening, so people don’t see all the work he does. And so the really good DBA, really proactive one, gets overlooked because everybody else in the company doesn’t see the fires that you’re preventing. And so they think you just sit back there and do nothing all day. Learning to speak up for myself and to publicize the things I do definitely was one of the best things I had to learn how to do.

Steve: Yeah, interesting. Yeah, I can see that how the firefighter reactionary DBA can say I fixed these ten crisis situations, where the proactive DBA it can be more challenging to say that I prevented these ten could be crisis situations, right? That’s really what it comes down to. That’s the difference between the reactionary and proactive. Ok, cool.

Carlos: Robert, our last question for you today. If you could have one superhero power what would it be and why do you want it?

Robert: If I could have one superhero power I want to be able to freeze time.

Carlos: Oh, freeze time, so almost like be able to walk through it where nobody else is moving type of thing.

Robert: Absolutely. Time is frozen for everybody else that I may be able to walk about and do things.

Carlos: Yeah, very handy when the boss is in your cube, right? When is the database is going to be up?

Robert: Exactly, so I think goes down and people bugging you, you could freeze time and get it fixed and then unfreeze and say, “It is fixed.” “What do you mean?” I think the challenge with that power though would be keeping it in the spectrum of superhero power as opposed to super villain power.

Steve: Yes, very good point. So many of those types of powers could be abused easily.

Carlos: That’s right. Well, awesome. Robert, thanks so much for coming on the program today we do appreciate it.

Robert: Thanks for having me guys.

Carlos: It’s been great.