Lessons

Lesson Info

Algorithm Questions

I am super excited to get back into my next topic on, and I'm gonna dive into the technical stuff so we'll talk about today is to talk about that without the algorithm stuff. So you dive into that and get into coding. And should you be using a whiteboard, why do we even used by board? Why why would we do that? And talk also about their programming, I reckon, and bring up a guest speaker to dive more into that first of his kind over them. So the reason why these that could ask you one big reason is that people tend to do good work. And it's something that I've seen when I've done acquisition interview coaching. So it coach startups tell them, prepare for the interviews they get when they're gonna enquired. And what I've seen is that when I interview somebody unlike Wow, this person seems to be really bright when it comes to Al Green questions. When I go back to their CEO or manager, that actually tends to be if I ask the right questions. That actually tends to be pretty representative o...

f their actual intelligence, Family says it comes to as least as it refers to coding. So, and I think that knowledge people tend to do pretty good work is that as the company's needs change, they can you know the needs for picker language. They're not as like tied to saying, I'm just a Java coder, only a Java code. I will never learn anything new. They groan, adapt with the needs of the company, to take on more and more challenging things and to learn new skills. They can be very effective if they're done well. And one reason also, that is kind of a little strange. But when I can make for doings, Malcolm interviews is that if you're a startup and you think one day you might just get acquired by one of big tech companies, well, they tend to do those interviews when you get acquired, so it'll be a lot easier. That acquisition will go a lot more smoothly, and it will actually happen. If I knew the hurdle that they're putting you through to get get acquired is the same one you put your employees through to get hired. I'll just go a little more seriously and you know, there's lots of questions that don't happen because the employee base was not able to develop in the interview. So just when you think about if you are much smaller companies setting up your hard process that said, they they can be very effective. But some companies, some interviewers, don't do a great job of them. One really common really, really kind of problem is they ask questions that are too easy. Eso asked a very basic question, like removing duplicates someone array. Not really saying much when you ask a question that easy. So that's that's probably most common issue a seed company Asking questions are too easy. Uh, another really common issue is an ah ha moment. So a question that relies on a single bit of insight. So one good rule of thumb here is, if you're asking a question and there is a hint you could given a candidate that would substantially change their performance. That's probably not a good questions, probably relying too much on getting that one peace. Even if that one piece is indicative of their performer of their actual skills, it's still just one data point that you're relying on too much so ideally, questions. You're perfect question has a lot of different hurdles and thoughts and different ways of solving it, and then really known well, really well known problems or patterns. So if you're pulling all your questions out of my book, you know your cannons are also preparing for that. So you're not really seeing. Are they great? You're saying? Are they performing? Are they pressuring? Well, so you and that also goes for certain types of patterns of problems are very well known. So even though the problem may not be well known, the pattern ISS so a good example of this is memorization problems. It is a very, very well established pattern. I have seen people have prep people where they're really struggle with it. I ask him some problem that's based on memorization. They do terribly the next day who do great because the pattern And once you learn that pattern there, good Ah, and you know it's still somewhat indicative of it of their skills that you know, the really, really bad people will never get that We'll probably never get that great but huge variable. And there is how much have they prepared that pattern? So those are the questions that are not gonna be very effective. Best thing to do is, you know, asking the right questions is a really important thing. Probably the most important. I think that the most common thing can companies do wrong is there just not asking great questions. Another Nothing is being really, really nice and friendly themes of the questions that start to get very intimidating for candidates. And so you know, I always advocate for being positive, reassuring and all that, but with these kind of questions, it becomes even more important. So being really, really positive, assuring when possible, it's nice to stay relevant. Eso You know, if you can pick questions that are kind of being kind of theme along the lines of what your company does, that's nice. It kind of shows candidates that, Hey, this stuff is relevant because sometimes cancel, walk away Being like that God, this interview's had nothing to do with what this company does. So nice if you can pick it being relevant. But that said A, I don't make a big deal of this. I think it's really important that you find the right questions and you find questions that are really representative that that you have a lot of different like hurdles to jump through it to solve the problem that are sufficiently challenging that don't require also, So I have all these other things that I really care about much more than whether not the problem appears at surface level to be relevant to the company. Um, I would argue that assessing someone's problem solving skills is fundamentally relevant, even if the question didn't appear to have anything do with the company's product. So it's nice if you keep them relevant, but I don't make a huge deal about this. It's much easier if you really want to bring make questions relevant. It's much easier do that with design questions. But you know, sure, it's a nice thing. Wouldn't do it. Coach can. It's through problems, um, so help them do their best. We'll talk later about what that exactly means, And, as I said there, will you find questions that have a slower in a fast solution? Ideally, now, as I said in another section, no questions perfect. No process is perfect, so I could take my questions. I can say I really like these questions in these or why, But it doesn't mean they don't come with certain loss. So just be aware of that. And, you know, these are the things I like. These things I want to find, Uh, some questions will be good, even though they don't have cover all those things. They're just not perfect. But nothing is so just be aware of that. So I want to talk about what the right questions mean. So medium toe to challenging. They can actually very very challenging. They could be kind, medium difficulty, but what you don't want to have is easy questions. So where would the kind of benchmark I would give here is if the candidates who are good enough to get hired generally no, the optimal algorithm within a minute or two that you are those good candidates That's too easy for problem. It should be a problem that the can, it's who are good are still taking at least 5 10 minutes to solve the problem optimally. That's that's where I would say a question of sufficiently challenging If it's much harder than that. That's okay to just give more guidance along the way. Multiple hurdles so you don't want to rely on a single bit of inside a single observation that Canada has to dio on unusual questions. So moving duplicates Marais. No, we should be asking questions far, far too common, Um, finding. If a link list has has a loop every so often here has seen that question. It's just even if you can argue, that's theoretically. Good question. It's no longer good question because far too widespread, I'd ours are argue that question is not very effective because it actually lies with a single insight. You have to know certain little trick, and that's all you're really saying. And so it's. It's not a great question that protected anyway. But make sure your questions are unusual and then avoiding obscure knowledge. So and this is something you really need to find as a company. What knowledge do you think it's fair game to assume that candids have and what is not? Here's my list. So you have for data structures. You have a ray list. Hash tables, trees and graph link list. Saxon cues, the basics. Core data structures for algorithms. You know, the too common sorting algorithms. Breath first up for search binary search on and then for concepts they go time in space, I think is pretty important people to know. And rickerson the idea here in the way that I had come up with this list is these are the topics that do. You come up over and over again, interview questions so it's not shouldn't be shocking to account to see them. But it's also the stuff that I feel like If someone has a CS degree, even if they're 5 10 years out of school and haven't looked at this, I, they generally will not have for gotten any of this. Maybe they'll forget some of the details of starting, but they'll know they'll still know what sorting is itself. So my basic idea of how one sorting out they'll still know, you know, hopefully what hash tables will still know what a binary search trees. So what I don't wanna have is questions that require somebody and no algorithms that, yes, they might have studied in school. But if they didn't study in school or if they you know, if it's been a year or two since taking the class, they won't remember. So some examples of that are things like Dykstra's algorithm. I've had people argue with me that, oh, it's not that hard to learn. It's not, but e don't want to know somebody that has ability to learn it or somebody still like I want to know if they're smart, there's good at developing things on their own. If your question realizing that knowledge or not, you're really just seeing it. How much of the studies Same thing for you to be? Details a tree balancing fair game to expect that somebody knows that trees can be balanced. I wouldn't expect them to know the details of this top electrical sort. It's actually fair game there to ascend. To derive that. I just wouldn't expect them to. No it. And there's a big difference for their deriving. It is not knowledge. It's taking what you know. Taking the knowledge of this is what a graph ISS and pick picking on new album. The fact that's a famous one. It doesn't matter it, but it's absolutely game. I've asked you to some flavor of driving that what's not fair is assumed that somebody would have that knowledge and then you know things like a star. So I get in ideas don't require knowledge that people will have for gotten, uh, now for people you're gonna interview locking without CS degrees. If you if you if you don't expect the one thing they say is take that. Put the best light again, give us out candidates. Tell them here is basically the kinds of concepts we expect you to know. Here is the kind of things that interview questions are often based on. So no one no one should be surprised on also keeps them from wasting time, learning a whole lot of stuff That's really beyond the scope of what you're gonna ask and walking away thinking like they had to know a lot more stuff than the good. So give us two cannons. Make sure they kind of know what to expect here and then, Okay, make sure the interviewers are clear on that as well. Let's put everyone on the same page here. Next thing is, make sure you're actually coaching Candice to the problem. So we saw this in some the mock interviews, but help them for so lot of the stuff. When I'm prepping somebody and when I'm you're interviewing somebody, things that well The things that I tell them to dio is you know, I have this seven step chart. You couldn't get it cracking the code interview dot com, and it goes like this first new. Make sure you've heard the problem. Make sure you remember all the details. Second step drawn. Go to a light board. Dry, big, drawn example. Make sure it's a specific example with, like, concrete numbers. Whatever in there, make sure it's large. Make sure it avoid special cases. Come up with a brute force our them but don't code. That brute force just stayed at ST the runtime. Then, you know, try to optimize it. Spend some time optimizing it before you start actually diving in. Once you have an algorithm, don't dive into the code. Make sure you understand exactly what you're about to write before writing it. That step five Step six is to actually write code. Step seven. It's a test. It's not, you know, nothing too shocking here, but seven steps. These are things you can actually one. You can actually provide that hand out to candidates, but also have your interviewers walk people through that way. If you know a candid goes up to the whiteboard, and they immediately tried to start writing code. You know that you have seen this a whole lot. I ask a question. Somebody immediately says OK, and then they start writing this for the I know that they're not going to do well and they're just gonna waste their time here. I'm not. That doesn't mean they're not. That doesn't mean they're dumb. It just means that there don't know how to be a good candidate. So stop him. It's This is also I say, it is on us interviewer to do that Time management to figure out to tell the candid don't code yet don't I wouldn't really hold that against Candace. So coach them through problems. Give them hints as necessary. Um, you tell them to draw an example. Tell them if they're examples are large or really small, and tell them to make it larger. You give them a more effective example if you want. You want to be careful to not get too overbearing and constant dropped them, but giving them some coaching along the way. Is it useful thing to do if they seem to forgot forgetting for gotten stuff in about the problem. Remind them of that stuff. You're not. Some is not bad because they forgot they forgot about the data being sorted. You mind that kind of stuff? Um, stop him from writing code too early if you that both goes. If they assumes you state the problem, the try driving to code. Stop them if you don't, I really believe that's kind of your fault for not stopping them, but also, if they cover their algorithm, they haven't often. When you agree to start coding and it's clear that they don't really understand their album or deeply, that would be a good time for you to pull them back and say, Hey, let's walk through this again. I don't think you get a lot of I don't think it says much about the candidate. Oh, then you know, not knowing the art of interviewing If they get hyper focused on writing code, I think that's much more of a reflection of the interview process. They're like, Oh, no, Oh, no. I need to get code on as fast as possible. And so they try to do that and goes poorly. I don't think that means they're not a good candidate. It is jumping in super early to start coding during an interview process. Do you think that that could potentially be characteristic of filed? They will approach a problem at work or after they've been hired. So it's a good question I can it be characteristic of That can be a signal. Absolutely it can be. But I think it's much more a signal of how that candidate conducts interviews. It's much more a sign that they just are nervous and they like that. I don't think you're gonna find a super strong correlation. I don't think find a strong correlation that I had hold it against a candidate. Uh, it's more likely the candidate has just heard something like heard these things I like. You have to write code and they interpreted to doing it immediately or, you know you have to. You want to get a working solution on as fast as possible and then took for that to actually go and do that. I think it's much more reflective of that. Um, And then, as I said, you know, your responsibility is to manage the time, so you know, I get, you know, do very, very, very, very regular, twice a week interview, prep talks, talking to candidates and really, really common question I get is okay. You said most interviewers ask two questions, which, by the way, I don't actually think you have to do. But this company that they're prepping for most interviewers asked two questions. They say. Okay, how do I manage my time? 20 minutes. The first problem, the 20th 2nd one that should be on the interviewer to say, Please go write code. Don't worry code yet. Let's move on to another problem. Your responses interview is to manage the time, and you should have that expectation. Um, phone interview. So I talked a bit about this, but just as a reminder. Don't go easy on the on the phone. I don't think it's the phone interviews. We're gonna set a really low bar. It should be representative roughly of the on site interviews. And then the major thing I do is just don't do questions involving diagrams. System design problems really don't work well over the phone. Uh, when you coding questions, I good. I do those over the phone. Ideally, you something like coder patter, Hacker rink. Why don't you? Don't ask. Combining research question. Just It's just really annoying. Doesn't help anybody goes and link list or raise hash table strings. Things like that, things that are linear, uh, evaluation. This is really, really important. And it's unfortunate how many interviewers don't really understand this. One question I get when I'm helping coming on their hiring process is they want, so they want me to give them a list of questions that I like. That's fine. I'll do that. Then they want these metrics. They want me to tell them this interview. Getting to this point in the problem is this score, and if it's after 15 minutes, is this is 20. I refuse to do that. I don't think that that's the way we should be thinking about interviews. The ways we think that evaluation is not did they solve the problem? Wasn't correct. Was it good enough for the real world? What you should be looking at is what is their performance telling you about If it takes you 20 minutes to solve some problem that most good cans Consol vin 10. But that's because we had this whole discussion about how hash tables resolved collisions. I cannot possibly say you are a bad, softer engineer because we had that conversation. One. I had that conversation with you. So it's my fault if I don't want to hear it. But secondly, that's actually good sign. Not a bad sign. So don't get tied up in the minutes. And is it correct? Is it not all this other stuff? Like, what is it telling you about them, Do they? You I'd rather you actually go on your gut with you. Try to back it up with evidence, but I'd rather you go on your gut of like, did they show strong problem solving skills? The way I think about this often is how hard was it for me as an interviewer? Did I feel like I had to relative to other candidates on that same problem that I feel like I had to spoon feed you the solution and like each little step did I feel like I could give you a little hint and you're basically will pick up on your own? But I feel like I didn't even really have to give you a lot of hints. I just had a don't you here and there or did not have to do anything that's kind of way I look. It's relative to other camps on the same problem. How much did I feel like I had to actually give you the Al Gore? Ah, now this requires calibration. It's really important to know that I I have a lot of interview experience so I can ask a new problem and have some idea of what the poor mechanics performance means. But it's some idea it's not. It's not nearly as good as one of asset problem a dozen times. So be aware of on calibration, it means that your data is less reliable. Um, that means the interviewers should not be constant switching questions all the time and viewed. Few interviewers do that, but also means of the early time that interviewers start asking questions. They're not going well calibrated, and their analysis will be a little bit less reliable. And that's when the interviewers should be putting in their feet back. If they're not calibrated that question, that's something that they should be signaling to the hiring committee or to whoever is looking at the performance. Next thing is so I won't talk about some bad questions here. So question. So I found a lot of time on Cora and one of the somebody asked me. You asked me to answer this question. I was like, Is this a good interview question? The question was, How would you implement Rand five from Ren? How infantry and seven from an. So this means you have a random number generator that returns there integer from one and five. How do you use that Teoh return around a number between one and seven on it. So I gave him this long thing. But why this? I don't think it's a good question and for a number, reasons. One. It's in my book. So if it was a great question, it's, you know, little A sophomore from that perspective, lovable have seen it. Now, Um, the other thing is that although it doesn't really require a lot of knowledge about probability, it feels like it does. And I've asked question questions that feel with their basin of animus. Um, and there's really nothing you know, people need to know, like, how do you do something with 40% probability? But it was also inquire a whole lot of, like, probably theory knowledge, but you should be aware that it intimidates people. Okay, I would get a lot of people who push back and say, You know, I really haven't done much. Probably they're just totally fine. But again, be aware of the weaknesses of questions. That's a weakness, a problem like this. But the more important reason why I think this is a bad question is requires this Ah ha moment. So one of the things that's useful to know is that you, once you realize it becomes a lot easier, you cannot do this term. Mystically, there's no way you could do 1000 or whatever ran five calls exactly that many or no more than that number and guarantee that you get around number, traitor and a number if you want us that there's no way to treat deterministic Lee. So you're gonna have no matter how you solve as long as, well, correct, always gonna have some probability of repeating infinitely. And once you know that once deterministic solutions are off the table and you say, Hey, you couldn't do it, even if it might theoretically loop forever and ever, ever Once that's off the table. You, you know you can do that probably comes dramatically easier. And that makes it not a great question, because it's that Ah ha Moment. Someone assumes that has to be deterministic solution, which is not a crazy assumption to make. They will not able to solve it if they don't. If they know that they don't have to do that will be much easier. So in too much randomness. And then the other problem, which is kind of funny. But something be aware of is this interviewer who asked me the question and then was defending why I was actually gonna question, uh, disagreed very much with my assessment that you can't do it deterministic Lee because he apparently had solved the term mystically which mathematically is actually possible. And so he didn't have this problem dozens of times and had completely the wrong answer. Totally the wrong answer. And that is funny. But it's also really sad because how many people that he reject, we're not able to do this thing that he apparently can't do. Um, but also only I've seen that's unfortunate that a lot of companies, when I look to their question databases, and I look at the solutions that they took up. I'd say about 10% of the solutions in the question databases Air action correct in some way. Maybe they have the wrong run time. Maybe the solution is totally broken. Maybe it's not as optimal, but it's incorrect ascension, somewhere other. So be very, very aware of that that often times these are not actually correct solutions. Eso you have a question database, You know, make sure you have a way of, you know, make sure the problems are correct. But also make sure when you're setting for across database that interviewer people have a way of realizing I've like commenting, Hey, this isn't correct. I'm sure interviewers looked through, and some realize that's not right. And they never went around. Took the effort to go update that other questions that are bad things like, you know, implant a stack with a single A linked list. Uh, this is this is just very, very common problems. First order of words in a sentence that's like, really common. First of all, it also is like very like a trick reverse entire sentence, and then you walk through the reverse each word and then the order of our universe. All the characters in the sentence first. And then you go through a reverse. See the characters and each word. Little trick. Ha! Now you got it. It it was too much. Yeah, maybe somewhat indicative of awarding really well done question. But really, it's Ah ha moment merging to sort of raise for two. Well known, not gonna get a whole lot of great data here finding two buckets in a string given, ah, given a book, find unique wars and recurrences just not too straightforward. Um, you place were faces with some of thing. Not just not hard problems. So you want things that are hard? It doesn't. You don't have to have multiple, totally different solutions. But you do want to have solutions that you can improve to getting more and more off one that So here's an example. The question I like. So you have two strings, One smaller, one bigger final permutations of the smaller string within the bigger story. So I like this for a couple different reasons. One, it's pretty challenging. Ah, it's even the best candidates. Will it be unusual? Not unheard of unusual to have accounted. Who with no help, could solve this in 30 minutes. So even the best cancer going to need some help? It's fairly challenging, but pretty much everybody can do something so almost everybody can get a brute force algorithm. Maybe not in Flint. It all but almost everyone come with an album that's like, Okay, generate all the permutations and then surcharge single one, Almost everything get back. And so off the bat, I'm getting some sort of signal. If you can't get that, then you know we have some problems. You can't drive that, Um, but it should get you some signal. You know, I'm making cats feel OK about the process. They're not sitting there flustered for 15 minutes cause they can't get anywhere. Got something. So it's good from that perspective. There's a lot of in solutions, or there's a lot of different, like steps. There's multiple authorizations you make, and so if somebody stumbles on one part, I get some extra data points. But if it works, uh, one of the things I really like about this is that there's no knowledge somebody really needs to, uh, this require you. I can ask somebody who has no CS to be no background and data structures, Really, all they need to know are like strings and arrays. And if, you know, strings and arrays were bigger problems, right? So I'm not I'm not gonna end of ruling out if I'm willing to consider people without CS degrees which the vast, faster, two cases, I should be willing to consider that I'm not gonna rule out the people who don't have the background. I'm not gonna make people people feel intimidating, intimidated because they don't quite feel good enough with this of that data structure. And so I really love questions that having at least some questions at my disposal, which don't require any complex knowledge. It's also great for a phone interview because this is one question where there is really no, like, it's all linear. Aiken, type this in a text editor, whereas a binary search tree question I can't. So the great questions, that perspective that's not super well known. I've asked us too many candidates. Almost almost nobody has ever heard of it before. So I liked that perspective is what? Another question. So this is the question I put up earlier, um, that we didn't mock interview as well, given a list of people with their birth and death years. Find the high find the you with the highest population. So I find that actually highest population of all time. So again there is a brute force out for them that is not completely, completely, completely brain dead. Simple, obvious. But it's pretty clear. And then we have a lot of different optimization is you can take. And so again, I get a lot of different data points like the past. Question. Nothing you need to know. No, you don't need to have have a knowledge, minor surgeries right like that. So it's a great question that perspective not super well known. Um, so you know, I like to have these questions that are hard, and this is a little bit using the previous problem, but hard. But the knowledge isn't heart. The album is hard enough. The knowledge That's a question I love to have. So one thing I want to kind of point out here is that you I push a lot on hard problems because the easy questions this when you ask an easy question that is the awful album is pretty obvious. You think about this as like, if I gave all of you in arithmetic test and not everybody would get But the fact that you got a 90 or 81 now put in the bottom 10% of the class and it does not mean you're bad at math. Just means you mis read something or you made a little type of right. That's one of the easy questions. They cluster all these performances together, and then the bottom 10% is just some randomness. Uh, so you're gonna be asking problems that are supposed to be about thinking. Make sure that they actually require it candidate to think if you are not going Teoh, you know, if you want toe, if you don't, I think people really need to be great. Like algorithm problem solvers Don't don't say. Therefore, I'm gonna give them, have them code. You know, Reuben too, because from an array on the white board. Okay, if you really are saying I just want test, make sure they're coding is competent, then you know what? Give him a computer