I see a lot of people in the comments describe triplebyte as a “market leader” and discuss how good they are. I can’t believe how gullible everyone is especially on a startup forum. Let me spell it out to y’all, nobody pivots and changes their entire business model if they are crushing it. So the implication is that triplebyte are not crushing it. So, this probably has gone the same way as lots of other tech startups. They’ve raised a bunch of money spent it all on ads , distorted the market for a while (usually just the advertising market) and now are on the way out. No matter what fluff or spin they put on it, that’s what’s happening.
Fair comment. I certainly don't think of us as a market leader. LinkedIn is the market leader. Honestly, the last year has been pretty hard for us (COVID and the problems with our model that I talk about in the post).
That said, I think a hiring process that puts engineers in control (no ghosting, get data on when a recruiter looks at your application, search ranking by whether companies lie to candidates) is something that should exist. I want to try to build it!
A small bit of feedback here. I tried TripleByte a couple of years ago and the experience was really polished. I passed the interview and got great detailed feedback which I really appreciated.
However, I dropped out completely because TripleByte wouldn't let me see companies without entering in a desired comp number. My recruiter at triplebyte said I could just put in "$1 or $1,000,000" to get past it, but that just feels like the same kind of corporate HR bullshit that people have complained about for years. Just Google "recruiter won't proceed without desired salary" and you'll understand how many folks dislike that question.
The experience made me feel that Triplebyte wasn't interested in putting me in control of the process at all if they have such a user hostile requirement and require me to work around it by putting in fake numbers. I understand that many people will have no problem providing this info, but I prefer to see what companies offer me because it tells me how they value me and the position they're hiring for.
You're totally right. If we'd been thinking in this mode when that particular requirement was built, it wouldn't have been. It isn't required today (and IIRC hasn't been for a while now).
I never, ever, ever give anyone a number until the salary negotiation stage of the interview (ie. after I've been given a job offer), and even then I only give the company a number as a counter to their offer. I always do this negotiation myself, and never let anyone else negotiate for me.
Almost all recruiters have been fine with that, but a couple haven't. To such people I wish a good day and go my own way. There are plenty of other recruiters in the sea.
On hiring side we screen folks out pretty quickly who can't discuss money.
I used to be much more relaxed about this, but if you get burned doing a few rounds of interviews and it was a waste you wise up pretty quickly - teams do not like time wasted.
I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons.
We're happy to talk through the different positions, levels, progressions and comp ranges for them all. But you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward.
You say "I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons."
The same goes in my experience for companies which insist on talking salary before the job offer. It's a red flag.
"you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward"
Of course I have a clue, but I'm not going to put myself at a disadvantage in the negotiation by revealing it before I've been given a firm job offer and heard a number from the company first.
Also, in my experience recruiters are the ones who push hardest to hear salary expectations up front. They know it'll give them a leg up in negotiations, and some of them are even paid specifically to get that number (as one recruiter was brazen enough to reveal to me).
> On hiring side we screen folks out pretty quickly who can't discuss money.
That’s because it benefits you as an employer.
I’ve been on the hiring side and I know there’s a budget. If someone is too high you want to push them out right away.
My flatmate in Singapore about 6 years ago was looking for a new job. Her salary was about 2200/m and she said the recruiter kept asking about salary.
I told her not to discuss it. She doesn’t /need/ a new job. If the company really values her then they will give her an offer. They have a budget. They offered her around 4600/m.
If she told them she was on 2200 it’s guaranteed they would have given her only a small bump.
Companies should never ask for salary info. Because it’s only used to get cheap labour.
Why can't you simply ask the candidate "We offer $min to $max for this position, is that acceptable?" That lets you and the candidate know whether proceeding further is a good use of time.
I've been a hiring manager in tech for a long time now, and recently as a candidate. As has already been commented, all companies have a budget. They may be able to increase that for the right candidate, but there is always a range, why not disclose that? As a candidate I know I'm not wasting my time, and the actual during the actual negotiation, its then that the 'value' of the new candidate to the org can be assessed. Pay people what they are worth, which is of course a reflection of their experience, talent, and attitude.
Indeed as a candidate, I don't want to waste my time either.
Your clients know their budgets. Why not just be upfront about that to the candidates. Tell us your budget or reasonable expected range your client can pay. Then if (when) we find that it's well below what some other hungry companies are paying, we can save you and us time by passing.
If you know your client is paying low, then just be open about it. Something like, "Ok, this client is a bit below market (and tell the upper limit they will pay), but they have these good things to offer." If the client has no good things to offer, and the pay is low, then you're still likely to not seal the deal with a candidate no matter what. So just cut the game and get to the point.
Giving a number early on in the process only weakens my position when it comes time to negotiate, waiting for an intital offer to counter is in my best interest. I apologize if that makes me a 'not great hire'.
If you're happy to provide a range, great, if I'm still there doesn't that generally indicate that the range is in line with my expectations?
I guess your comment kind of makes sense, I can see how corporate might interpret 'this person looks out for themselves' as a potentially bad hire.
The problem is that companies use any salary information you provide as a candidate to adjust their offer. A lot of companies that asked me what I made at my previous job (even though it is illegal to ask in Germany) offered me just 20% on top of my number.
This makes me think that all jobs position should have rather narrow target ranges which are stated up-front. Then you can discuss if you are too high level/low level for that job and range.
No it doesn’t. The company is fundamentally in a much better negotiating position (they have much more data on salaries) and carry way less risk than the employee (a company has many employees, each employee has just one job).
In each negotiation about a position, the company is negotiating only about one of its many positions; the candidate[1] about what will (usually) be their only job. And companies usually, AFAIK, get a lot more applications[1] per position than each applicant[1] gets offers per job search.
So no, AFAICS you are wrong and the original thesis of "the company holds the advantage" is correct.
This is great actually because you avoid wasting candidate's time, they can move on quickly to better companies that have a more fitting interview approach.
A particular problem with doing that is discussed in the post; you risk everyone's time being wasted if the company will never be able to meet your target.
Why should it always be the candidate's problem to reduced that risk and not the companies?
That time would not be "wasted" if the company had done due diligence on what the position and candidate were "worth" and revealed their target (or target range) to the candidate.
The real reason, I'm sure, is that companies think they hold the upper hand in the negotiation. And a lot of time with a lot of candidates that's true. It's clearly _not_ true for the class of candidate who's realistically got a $300k+ target. You are totally not going to successfully headhunt anyone in the top 1% or 5% of FAANG engineers if you try to dick them around with recruiting power imbalance power plays. _They_ don't need your bullshit. They will totally walk away from whatever time you've "wasted" so far trying to recruit them. You need them _way_ more than they need you.
(And I'm pretty sure that generalises a lot further down the experience/renumeration food chain than most devs realise. Fresh grads or devs with only a few years of experience might need to play the stupid recruiter games, but most companies trying to fill a mid level or senior role need the right candidates way more than those candidates need any one specific job. You can and should be able and willing to walk away from any potential job offer over stupid games like this...)
1 - I get real-world interview practice, and that always helps me in future interviews. Many companies ask the same or similar questions, and the more I'll be ready for them being asked again. In addition, when I'm surprised by a question I'll research it thoroughly when I get home and won't be surprised by that question again.
2 - I get a bit of an inside peek in to how the companies I interview at work. That's a privilege not afforded to many outsiders, and how some of these companies (some of which are world class) can and has helped me at other companies.
3 - A company might bring me in to interview for one position, but after the interview might determine I'm better suited for another. If I didn't interview with them they'd never know.
4 - As I interview with people I get a chance to make a good impression on them, so even if they don't wind up hiring me for that position they might want me when a different position opens up (whether at that company or another company the people I interviewed with moved to).
5 - By letting companies make the first offer I myself get a good salary survey of the minimum what all the companies I interview with are willing to pay for someone with my qualifications in that position. Of course I try to do my own research ahead of time, but getting salary numbers without getting a job offer is not always possible or reliable.
6 - If I blurt out a number first I'll it may put me at a disadvantage in the negotiation, as they'll just start the negotiation from what I say even if they would have been happy to have paid way more.
> I get real-world interview practice, and that always helps me in future interviews.
I hear this a lot, but how much interview practice do you really need? After the third or fourth set of interviews, they start to really blur together for me.
None of your points are false, but if you're casting a wide net and taking every interview without any idea if the company can even afford you, that's a lot of hours spent. The last time I looked for a job, I interviewed at only four companies and spent an average of probably 5-6 hours on each, and it was exhausting. I can't imagine making a habit out of it while also working full-time.
I agree it's exhausting. And it can feel very bad and demotivating for the whole thing to end up without an offer or some half-assed "culture fit" "explanation" that is likely copy-pasted from an online script collection.
But, in my case, doing many interviews helped me understand what negative signals I am emitting during interviews [according to the interviewers]. There's nothing like an outsider's perspective, even if you disagree with it.
---
Example: in one of my last interviews I was told that I am too "cocky" and "over-confident" because when I was given real-time modifications to a homework assignment (that I completed before) I was smiling and saying "oh, that's easy!" and proceeded to make the changes live in my editor -- while sharing screen.
This feedback left a very WTF feeling in me. Dude, I am (a) enthusiastic about the task and (b) have enough experience to show you, in real time, that I can modify the code right there and then to suit the new requirements, and (c) chatting with the interviewing team while doing it... and all you can gleam from that is that I am overconfident and cocky? Seriously, get a grip.
For the record, I strongly disagree with their take. But it gave me a very interesting perspective -- namely to scan for the more introverted / likely-insecure people in the interviewing dev team and try and act a little more modestly to not rub them the wrong way.
It's a fact of life that most people suck at interviewing -- as I hope that I demonstrated here. One of the guys in that team simply dislikes certain kinds of people and he lets this go against his better professional judgement about my abilities. Not cool, right? Especially when you also take into account that the CTO who also attended all interviews agreed that my tech acumen is top-notch and they seriously couldn't catch me with my pants down no matter what they did.
So again, this helps me gather experience and culture and makes me more adaptive in the future.
Does it hurt your ego? Absolutely. Does it shake your preconceived notions? Much more than one would think beforehand. Is it exhausting? Hell yes, sometimes after an interview I didn't want to do another one for a week.
But, is it also very valuable to help you improve your interviewing and negotiation skills? Yes!
That's why I did lot of interviews the last time around. And I'd do it again.
Yep. I won't get to the offer stage without us having discussed money. I don't want to waste either of our time only to find out someone's only willing to pay 70% of my bottom line.
For a big enough company, you can collect a few data points via Glassdoor[0], levels.fyi, etc to make sure that they pay in the range you're looking for before you even begin the interview process. For smaller companies, you're absolutely right, although I personally tend to give a very broad range[0] at the beginning of the process, and only narrow that down at the offer stage. That's worked well for me in the past.
The nice thing about the explosion of remote work due to COVID is that engineering salaries are beginning to normalize across the US, and there is now a lot more salary data available online, so it's easier than ever to see what you're "worth" on the market on average. This at least gives you an anchor when having these conversations, so you're not going in blind, and gives you a lot more leverage if they low-ball you.
[0]: Yes, I'm aware of the drawbacks of self-reported data, but it's still broadly useful in aggregate.
[1]: I'm making up numbers, but something like $120k - $200k, with the lower number being the minimum you'd realistically take. Usually recruiters are honest up front if they can't even afford your absolute minimum, but it still leaves you with a lot of negotiation room once the offer comes in.
"I'm making up numbers, but something like $120k - $200k, with the lower number being the minimum you'd realistically take. Usually recruiters are honest up front if they can't even afford your absolute minimum, but it still leaves you with a lot of negotiation room once the offer comes in."
Why even give an upper limit? It's not like you'd refuse $400k if it was offered to you, would you?
Most companies will naturally start negotiation near the lower end of your range, even if they would have started way higher had you let them make the offer first. I really don't see an upside to this strategy.
Because if you just give one number, even if that's your absolute minimum, the entire salary conversation is going to be anchored around that one number, and you're going to have a harder time asking for 50+% more than that. If you give a broad range, it's easier to justify countering with something closer to the top of that range.
> It's not like you'd refuse $400k if it was offered to you, would you?
Of course not, but nobody has ever offered me that. :) In all seriousness, yes, you run the risk of your range coming in below what the company would be willing to pay, but in my experience (and those of friends and acquaintances), outside of FAANG, I have found that to be extremely rare if your range is wide enough. The companies that pay super well are generally big, and thus well-known to pay that much, and I wouldn't give them any salary numbers at all because I'll have a pretty good idea of what they'll pay me with a bit of research prior to interviewing.
Of course, anecdote != data, so YMMV and all that. This is just my experience interviewing at mostly smaller companies.
They uniformly underestimate big tech compensation by 50%+, because they've never clearly delineated salary from total compensation (and stock grants are a majority of comp for a lot of senior FAANG employees).
Luckily levels.fyi has fairly accurate data for those same companies.
> For a big enough company, you can collect a few data points via Glassdoor[0], levels.fyi, etc to make sure that they pay in the range you're looking for before you even begin the interview process.
Agreed though in practice that just moves the pre-interview conversation from a comp discussion to a leveling discussion.
Seems to me that you would waste a lot of your personal time this way. They know enough about me before I even step in the door and I want some idea of whether the money can work. I ask for more than might be typical and I haven’t time to waste on companies that just want to show me what a great place they are to be at.
conversely, I discuss compensation immediately. In the segment of the market I'm in, I've wasted too much time going through interview processes only to get to the end and find out they're incapable of even matching my current comp. I've found being candid more successful.
Yeah I think it’s worth giving at least a range if you suspect the company can’t even meet the minimum. Like “at this point in my career I’m targeting positions that pay from $150-$200k.” If they balk and basically say the aren’t authorized to offer over $100k, you don’t have to waste your time. This probably makes more sense with smaller companies where they tend to have more limited budgets.
In my country it's common for companies to have public salary range, at least for IT positions - see https://justjoin.it/ for example board, Linkedin recruiters do the same thing.
It seems to be terrible waste of time going through interview process to learn that they won't even come close to what you want. I would not do that to myself.
that used to be true universally. now only true for entry level at big companies.
at senior levels especially, you should know what you want. comp bands are informed by huge data troves now and any edge you may have gotten before is gone due to the fixed and tight bands. just state your requirements and insist on “turnabout is fair play” ie they need to tell you the band. do this during the prescreen and save everyone’s time. if you kill it in the interview you still easily can negotiate to top of band.
Semi-serious question, asking for an indeterminate person:
If your desired comp number actually is $1M/year, will that just flag you as a joke to Triplebyte companies? With post-pandemic stock price increases a lot of engineers at FAANGs are making more than that now.
> If your desired comp number actually is $1M/year, will that just flag you as a joke to Triplebyte companies? With post-pandemic stock price increases a lot of engineers at FAANGs are making more than that now.
For any reasonable definition of "a lot", no they absolutely are not. A lot of people with 2+ years of experience who've either been promoted once and received a strong annual review, or switched jobs as a mid-level industry hire are making in the 200s and 300s, a few pushing into the 400s. Much more than that and you're looking at the top 1% (or less) of engineers at the top companies. Not a lot.
Source: was a mid level engineer at a FAANG until I left late last year, knew seniors and principals and their comp. Nobody was making $1M/year.
If "quite a few" non-directors make more than average American cardiologists either:
A) your company is engaged in disturbing monopolism;
B) your company is engaged in disturbing labor exploitation and arbitrage; or
C) your company is engaged in disturbing environmental destruction.
Would love to be proven wrong. Am I?
Edit: option D) is that American cardiologists are not as exclusive and skilled a labor force as I would prefer to believe ;)
People are paid to create value. If they work at a company that reaches a billion people, then writing some software that adds $0.1 revenue per user per year is worth a lot.
But people do brag about it on Blind such that the compensation bands are well known. People also complain about cliffs when it turns out the company didn't want to pay them at a certain level.
Stock price for most of the FAANGs has gone up by about 2.5x in the last year. If you had $200K base, $50K bonus, and $300K stock compensation in 2019, you're now making $1M today. That's roughly L6/E6/ICT6 (staff) level.
FB in May 2019 = $177.47, today = $332.04, 87% appreciation.
AMZN in May 2019 = $1775, today = $3496, 96% appreciation.
AAPL in May 2019 = $43.77, today = $130.87, 198% appreciation.
GOOG in May 2019 = $1103, today = $2508, 127% appreciation.
That's all over the last 2 years. Market low was apparently Dec 2018, before the Fed started cutting interest rates, so the numbers are a bit bigger since then. They were pretty steady through 2017-2019, so anyone who got grants or refreshers during those time periods has benefitted from that appreciation.
They are making that on their 3-4 years vesting and refreshers etc that they can't sell. So yes, in past few years some folks have been making great to really great money especially if they have a bunch of equity in various vesting stages + get a big refresh / bump on a position switch to drive retention. I wouldn't say a lot but many maybe?
> Just Google "recruiter won't proceed without desired salary" and you'll understand how many folks dislike that question.
It's incredible how mileages vary. I recently had to strike companies off my list which demanded to know my last salary.
I offered them my salary expectations, but they would absolutely not accept that, and preferred to take the fact that I would not tell them the exact salary at my last position as a reason to decline to immediately stop the interview (with half an hour left in the call).
This wasn't a one-off either, I would say about 5% of companies (biased sample) try and get away with shit like this.
I get what you are saying, but on the contrary, I don't want to go through interview process as a candidate just to be lowballed in the end. If the range is not there, and often it is not, I am fairly good and not everyone understand how much this costs, I just don't want to waste anyone time.
I don't think this part is off. Why I never went to Triplebyte and will not go, I think doing tests is dumb and don't wanna do it. Like original commenter said, you can't change model that quickly.
There is a large pool of people who think that they are programmers, but are not. This is exactly what the infamous fizzbuzz test was intended to weed out. I personally know a top notch recruiter who credits her success due to learning early in her career how to filter most of the junk out. For example a "Perl programmer" who listed Microsoft Word as a skill was probably not worth looking at. (Note, this is an example from 20 years ago. But the principle remains.)
If you create an engineer focused site and DON'T find a way to segment engineers by ability, you'll find that your negotiating power is based on the average of your candidate pool. Which means that you'll discover why traditional recruiters kowtow to companies and not the other way around.
This is very true. We weed out hundreds of applicants ler opening on very simple questions... Think what is the difference between const, var, and let declarations in JavaScript?
The problem is we're all spending weeks and weeks on weeding out fakers. We've moved to a quick pop quiz right after we get an apply. This takes out 85% of the candidates (it makes me sad when I look at how simple the stuff is that weeds them out...). Two interviews are more than enough to land on a hire after that.
Our company is a conversational recruiting software company (you apply, we say hi, and screen candidates in real time and then hook the candidates up with an interview). We started doing the screening in our chat flow and it works really well for us.
Very good advice. The "gatekeeping" they alluded to is just very necessary if they want to have any differentiation from any other generic job matching service. I'm sure the triple byte team is aware of this but I think it's important to keep in mind just how much people work in tech/IT jobs, how wide the skill/qualifications range is and how spammy applications can get very, very quickly. Even positions that are "easy" to get as a competent engineer can get hundreds of applications, and since it's also pretty common for people to just outright lie on their CVs it becomes very time consuming to filter for someone who genuinely meets even just most of the job requirements.
& maybe I'm wrong but I thought the entire value proposition of TB was that their pre screening and testing process made the whole process much less noisy for your hiring managers. If you still end up needing technical interviews and aren't guaranteed that all the candidates you get have gone through some technical screening, what's the point?
AIUI, employers will be told whether the candidate took the tests, and if so what their score was:
> Going forward, our assessments are a purely optional means by which engineers can show their skills to employers (who overwhelmingly tell us that they trust our scoring), rather than a requirement for entry.
I think the idea is that an employer can still filter out the unfizzbuzzables, but has the option to ignore that for candidates whose stellar track record is a stronger signal.
I'm a little bit skeptical, because i think there are lots of people with apparently strong CVs who are still weak when you actually put them to the test. But maybe not at the very top end; i am really thinking about people with experience in various investment banks, which is perhaps less of a signal than Apple then Netflix or something.
If you want to put candidates in control, build a CRM tool. When a recruiter pings me on LinkedIn or emails me, I have a URL I can send them to with an intake form. No more calls wasting my time, I can see the company/role/salary/etc. up front.
The trick here is you need to figure out a business model where you don't take money from employers, because once you do its just the long slide to becoming a shittier LinkedIn.
Colorado now requires job postings to include something more or less like a "reasonable salary range". It hasn't been perfect, for example there are some companies who now just restrict their online job postings to say "except Colorado" instead of adding a salary.
But as someone living in Colorado it's been nice to know the salary range up-front more often and to have confidence that I can ask them for a salary range without getting the question turned back on me: "Well, what are you looking for?"
I think as they start enforcing things more employers will avoid colorado until kinks worked out.
If you have a janitor in CO, every job is an improvement and potential promotion. Having to first go to that janitor for every position is going to wear out FAST. Posting for the CTO position? Go to janitor first. Posting for CEO position - same thing - that's what is going to drive folks away.
In my experience this is worse than nothing, because the posted pay is so much lower than the companies are actually willing to offer. Positions posted for FAANG that I know are offering $250k+ total comp are posted with statements like “pay starting at $110,000 but varies based on your unique qualifications and experience”.
I'm unsure of exact enforcement and such but you could report them if you wanted. I also don't mind FAANG so much since there's so much info available for those positions on like levels.fyi.
The biggest strength for me is that I can ask them and they have to tell me. When I was job searching before I'd ask the recruiter and then they'd always ask me what my target is. Not showing your cards is like negotiation 101. Plus it's made fielding messages and emails easier. If they provide a range, I can just easily say "This position sounds interesting but unfortunately the salary range isn't what I'm looking for currently."
You're right that a lot of positions aren't great about this, I've literally seen "Pay starting at $0.01" or ranges literally $30k to +$200k. But I've seen it as a benefit in the negotiation process and it's just been easier to field recruiters.
> total comp
This law only covers salary as far as I know, so equity and such aren't included. Total comp is different.
I guess not, but this is also a law that applies to _all_ job postings for CO. I feel like a lot of people are criticizing this law when I see it as an absolute step in the right direction.
It's not perfect, but I feel like it's wildly helpful for a ton of people. It helps transparency for a lot of positions and I feel like it rewards honest companies. If equity and bonuses are a big part I think it reduces the overhead for talking about salary to clear the way to talk more about equity and bonuses.
It's also in the early stages and CO is the only state with a law that requires disclosure up-front to my knowledge (someone mentioned CA has a law, but they only have to answer if asked). It feels like a fantastic first-step to me.
Unfortunately it looks like companies can just state wide ranges (which don't help applicants understand the market any better) and they can also omit other compensation elements beyond the base salary component (stocks, cash bonuses). That makes the disclosed information not very useful.
We've talked about things in this area, but this particular framing is pretty interesting. We're talking about it now.
I personally like this proposal and am probably going to at least draft a hypothetical spec of it and see how it fleshes out. Thanks for the suggestion!
It's unbelievable that nobody has done this well - I and most people I know are tracking their own job searches on a spreadsheet.
Pretty obvious value-add for job searchers. Not to mention that having access to this data would enable tons of other product features. Shows that most services/sites don't care that much about the applicant experience.
If you're interested in comparing notes, I'd love to hop on a call with you. (We've got some bookable slots linked at the bottom of the blog post, but I can make time elsewhere if you'd like.)
I can't remember the last time I was ghosted. The problems you listed aren't problems that I experience. I don't care what part of my resume recruiters look at. I don't care what they do with it.
Sure, having metrics on how often a company ships code, etc. would be helpful, but frankly I have about 10 companies on my list that I'd work for, and I doubt any of them would give you this data.
What I'd really like is not having to practice leet code problems for 3 months before I interview. The older I get, the less motivated I am to do this. I've worked in senior engineering roles at big tech (FANG) with 3+ years tenure. I don't think I managed to not be able to do my job and fooled people for that long.
I've interviewed people I KNOW could do the job, but they didn't get the optimal solution, or were nervous, or N things in the loop, and we ended up not hiring them. There must be some incentive to create a system that lowers these false negatives.
I have had this exact experience with Triplebyte both as an engineer and as a hiring manager.
I simply do not care about things like, “what does malloc return”. I do not care what people know about bloom filters in Postgres. The 99.99% of web developers don’t have to know these things. I do not care if you can implement Tic-Tac-Toe.
It’s poor interviewing, which is the entire product they’ve been offering.
> There must be some incentive to create a system that lowers these false negatives.
I’m generally quite pro immigration but when BigTech goes to DC and bellyaches about how they need higher quotas because they can’t find enough workers it really sticks in my craw.
Reminds me a bit of the classic example of chutzpah—-the boy who killed his parents, then threw himself on the mercy of the courts, asking for leniency because he was an orphan.
> I think a hiring process that puts engineers in control (no ghosting, get data on when a recruiter looks at your application, search ranking by whether companies lie to candidates) is something that should exist. I want to try to build it!
This is a space that unions fill well compared to for-profit recruiting agencies. The organizations give engineers control and leverage in hiring, and can help them up-negotiate. They also allow employers to hire from pools of skilled talent.
Whereas with recruiting agencies, the customers are employers, not the candidates. Incentives are not aligned to give candidates control.
While I still slightly prefer the direct experience for other reasons, the recruiting agency experience was pretty good for me in its own way. More specifically, when it comes to salary negotiation. Let me elaborate.
People at the recruiting agency usually get paid a commission as a percentage of the first year of the hiree’s salary (not taken from your paycheck, but just in general). Which means, it is in their interest to get you as high of a comp as possible. Which also means that I dont have to bs around with the recruiter about my salary expectations or anything like that. I give them the upper range of what i want, they tell me either “sounds good” or “they are easily willing to pay more, so why not try $X instead” or “they cannot pay that much, but there is this other company that we work with that can offer you as much.”
And if you work with one recruiter consistently, it gets even easier, because now that the recruiter knows my total comp expectations, she pre-filters opportunities for me and only contacts me if there is a position she is aware of that pays at least as much as I want or more.
Of course there are downsides too, for example, the limited choice of companies that the recruiting agency is working with at any given time, so i still have to find some opportunities and interview with those companies on my own. But as a supplementary option, working with a good recruiting agency has been quite nice for me.
P.S. The type of a recruiting agency i am talking about is the one that is focused on a specific sector (in my case, fintech/quant), so it might be a completely different story for a “generic software dev” recruiting agency.
> Which means, it is in their interest to get you as high of a comp as possible.
Beware: this is not universally true. What you typically see is that it leads to closing as many candidates as possible. A recruiter can make more by closing 2 candidates at 80% of max comp than 1 candidate at 100% in the same time.
This is a well known behavior in real estate sales, which have a similar commission structure.
Agreed, but as you said, it all depends. I am not sure if it was specific to that individual recruiter that I have been working with or if it is just a general practice with fintech recruiters, but if my recruiter was going for the high volume, she had made a really poor choice deciding to continue working with me.
I am saying that because I’ve been working with her for at least 3 years now, and I am yet to take a single offer from the companies she set me up with (either due to them getting cold feet last second or simply due to me not passing some of those interviews or me declining due to some red flags with the companies). If it was all just about high volume for her, it would have been in her interest to drop involvement with me after the 3rd unsuccessful attempt to set me up with a fintech company.
>This is a well known behavior in real estate sales, which have a similar commission structure.
Even worse, real estate pays both sides so realtors are heavily incentivized to get their buyers to purchase from their own sellers so they can get both sides of the commission. Instead of paying them 3% to act on your behalf and do a bunch of leg work they get double and now they can do less work and aren't strictly working for your benefit.
Working on commission just leads to wanting to make a deal happen. Even ignoring the time aspect of it, if a recruiter has much better odds of making a placement at 80% max comp compared to 100% max comp, the 80% is still objectively better. Why risk not closing the deal when the expected return is negative?
Collective bargaining is inherently about eliminating competition between workers. That is, it creates a labour monopoly to balance out a labour monopsony. That might (might!) be a good thing for autoworkers. It would be an unmitigated disaster for software engineers, and the tech sector in general.
People who write code are unbelievably productive compared to just about any other job function, and software engineers (who we'll define as people who write code for tech companies) are even more so. Part of eliminating competition between workers is making sure that nobody works harder than anyone else. That would be devastating to the productivity of software engineers, which would make them less valuable to tech companies. That would lead to a decline in salaries, and general stagnation of tech.
Another aspect of preventing competition is seniority. One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top. A union would tell that kid, "sorry, you'll have to pay your dues as a junior engineer for 5 more years, then you'll be eligible for promotion. In the meantime, we'll give that job to cwp, he's got the required seniority."
Just think how this would affect tech companies. Sorry, you can't pay more to attract the best. If you're really careful about interviewing, you might be able to find some people with both skill and seniority, but it'll be tough because smart, ambitious people have all gone to Wall St or whatever industry has the best opportunities.
> Collective bargaining is inherently about eliminating competition between workers
A yet every movie star in the US is in the same union as B-movie and infomercial actors.
> Another aspect of preventing competition is seniority.
Weird, considering pro-athlete unions regularly help kids out of college quickly become multi-millionaires.
> One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top.
Weird, considering unions have been at the forefront of education, apprenticeships and training that helps kids out of school rocket to the top, the same apprenticeships and training that companies refuse to fund themselves.
> Sorry, you can't pay more to attract the best
Weird, considering SAG-AFTRA regularly helps movie stars up-negotiate multi-million dollar contracts.
Agree with most, but your valuation of coders is humorous. Save a few, they're not that special. Most are more similar to the factory line worker than other job functions.
As a candidate, I'm willing to put up with sometimes shoddy interview outcomes (e.g., ghosting) if my funnel is far greater and consequently leads to a more competitive offer, which is the end goal. A narrower funnel, that pre-selects for higher expectations of how companies engage in the process, may end up falling short of what I'm optimizing for.
Exactly, I don't understand all this moaning. You're only doing this every some years, you might as well try to optimize it rather than be picky, because it can make a big difference both career and compensation wise.
Idiotic HR does not correlate in any way with a worse or better company, at least in my experience.
That shouldn't have happened. Could you go ahead and email our support at support@triplebyte.com again (just so that you don't have to expose your data here) and let them know who you are? I let one of our support folks know to keep an eye out; we'll get you sorted out (and I'll check back here later this evening, so let me know if you don't hear something back quickly).
This is an admirable aspiration. Although, if I think about, the companies spend much more time with a recruiting/sourcing product. And the candidates change jobs like once every 1.5-2 years? So optimising for a candidate experience doesn't seem right considering they're not the customer.
Your point in the article about gaining bargaining power for engineers by making the hiring process an iterated game instead of one-shot is a good insight.
But the transparency you need to extract is mainly financial, that’s the big asymmetry. I think people care more about money than they say they do, and less about release cadence or test coverage or whatever than they say they do.
So triplebyte should be called “get me a raise.com”. I tell you what I’m making, smash all your tests, and you find me somebody who wants to offer me e.g $50k more or $100k more, and that WILL be the offer I get. And if the company reneges on that offer then it’s flagged to warn others, and if they do well by their employees they are rewarded as well.
This was our original theory when we started doing user research, and we were surprised at the degree to which it didn't pan out. We assumed (in line with how these things usually go) that engineer interest in filters would follow some sort of very lopsided power law, where a few filters dominated among almost all engineers. But in practice, everyone wanted something different: some people were excited by salary, some by tech stack, some by pair programming, etc. One guy went on for like fifteen minutes about how much he hated open office plans.
That said, the idea of "we will offer X to anyone with Y quiz scores" as a showing a company can make is pretty cool, and is definitely in a "company makes public conditional guarantees" space that we're interested in exploring.
I emailed customer support in April because your site would not allow me to change my current role to any value other than <Specific title at specific company>. Like, my profile literally has a dropdown menu where that's the only option. Nick O'Brien asked me for a screenshot of the problem then ghosted me. Today it is still impossible to change my current role on your site.
I am looking forward to progress toward a hiring process that puts engineers in control, such as by allowing them to accurately record their current role in their profile.
Do you think more of a peer based approach to recruiting would have any legs? I built something a little while back that was a kind of "keep track of the people you'd hire", and anyone on your list had the power to grant you visibility to various parts of their profile (current employer, whether they are looking or not, what kind of role they are looking for, etc). Never targeted recruiters as a potential user, but I guess that probably isn't a bad use case. I'd much rather get contacted by a recruiter (albeit a recriuter who was known to me) with "hey I see you're looking for X, let's catch up to discuss opportunities Y and Z" as opposed to the cold calls / linkedin messages that are the current status quo.
I used TripleByte to recruit people for a startup without paying, by asking people to sign up for the test and screenshot the answers and collecting the scores at the end. I already had an inbound funnel, I just liked your test when I saw it the first time.
In 2015, candidates with the same schooling scored about 20 percentage points lower than 2020, the last candidates I used the test for. The number of questions doubled, and then the test got much easier, by eliminating more challenging programming questions and replacing them with questions with giveaway context. By comparison, according to my data, about N=43, until about 2018 being a senior versus a junior in college CS programs is worth about 10 percentage points on the test; going to Harvard instead of Berkeley is also worth about 10 percentage points. In 2020, the last tests had no predictive features.
I stopped using the test, because it became too easy and too noisy to be informative.
I recognize some of the coded language in the blog post. There are definitely more lucrative opportunities in recruiting for DEI. I don't know if it will last. If you're still jittery about the public-profile-by-default thing, which by the way, was totally irrelevant and overblown IMO, this may not be a pivot for you.
One thing I see in the data is that at MIT, women and men performed the same, controlling for seniority. This wasn't true at the 3 other universities that produced enough data to measure.
That said, what really is the best way to hire candidates? I'm not convinced having binders full of engineers is special, there are almost always more candidates than jobs, at least 5:1, in every non-credentialed industry vertical. Anyone who has worked at a jobs (or indeed any matching platform, like the Common App or Tinder) knows that.
Then there's this long thing about asymmetries or whatever, warble garble about missing information... It has never, not once been my experience that someone seeking a subordinate role at a typical private company with preferences like "pair programming" or whatever have ever been better than someone with no preferences at all.
Maybe it helps to engage in the vanity of whatever trendy workplace trend is hot for whatever vertical. But like, if you're being intellectually honest, if you thought pair programming was important, you'd pair program at TripleByte, but you don't, you know in your heart of hearts none of that shit matters, so why are you putting stuff like that into your search system?
Indeed and ZipRecruiter are ad arb companies. They don't care. Private universities lead, not lag, DEI at giant companies, so it's hard to see how to compete against them in that core business. It will still come down to a real defensible opinion.
Do you have more valuable inventory than ZipRecruiter for DEI candidates? Who knows. What an uninteresting question. Apple also hires people who just make shit up on their resumes, I know two - though they weren't engineers.
A recruiter once told me, if you see 30 applicants for a high level tech job on LinkedIn, not to worry about competition, because ~27 of them were likely to be unqualified trying-their-luck applicants, mostly from other countries where the company was not going to hire, and mostly with no relevant skills or experience.
They told me the actual number of qualified candidates was usually just a handful or less.
I doubt this is true in SV, but I don't live in SV.
> going to Harvard instead of Berkeley is also worth about 10 percentage points.
That's interesting to me. Most reputable rankings put Berkeley at #2 or #3, and Harvard is usually around #7 or #8 (and US News puts them at #16!). I wonder what the cause of that discrepancy is.
CS program rankings are largely based on their graduate programs. The parent comment is almost certainly talking about candidates with bachelors degrees. For undergraduate Berkeley has a 16% acceptance rate and Harvard has a 4% acceptance rate. This will likely act as a filter in combination with the quality of the undergraduate CS program, which could be better at either institution but is hard to get a metric for.
I was specifically looking at undergrad rankings. The acceptance rate is not a good filter because that's the acceptance rate for the entire school, not the CS program.
From what I can tell, anyone who is a Harvard student can declare the CS major if they pass the pre-reqs. At Berkeley, passing the pre-reqs doesn't get you into the major -- you still have to apply for a limited number of spots, which generally means having a 3.8 GPA or better in your pre-reqs. So effectively both programs have the same acceptance rate.
Pedigree filtering is most often myopic elitism, especially in a business context. Anecdotally, I avoided MIT and Harvard because of Boston's snow and traffic. CalTech is Pasadena: too frick'n hot. I didn't study at all for the SAT-I and aced the math section. (The SAT should be more like the JEE.) I went to an expensive, "top 50" public university where I liked the area. Also, it was more practical and rigorous academically than most Pac10s (Pac12s now) and Ivys because they had something to prove (no grade inflation at all, they don't care if you turn in homework or not, not much market for homework and tests, and proctored exams). I also didn't do a PhD because of the economic disincentive: if I put in the work and the cost, I could do an MD, JD, or PhD CS but there is no added benefit.
Most clinical doctors have social skills and crystalized intelligence from domain expertise, but aren't typically mistakable for particle physicists.
People don't necessarily require the proper sheepskins to possess fluid, crystallized, and/or other domains of intelligence AND the skills, personality, and experience relevant to excelling at a particular STEM role.
Pedigree is mostly used for social filtering and business leadership board packing, but if someone wanted to create an elite monoculture of staff, lacking in cognitive and personality diversity, by all means, go right ahead.
PS. I. Don't even get me started about a car full of CS IITians talking about the JEE and high-placers who seem normal on the way to snow country for snowboarding and gambling.
PS. II. I bombed an Apple interview for a mid-career role by being too intelligent and too maverick compared to the group of compliant, I hate to say, yuppies. It was an interview panel of about 9 people and they were just speechless. The other time I bombed an interview for being too smart was about 10 years before that when I was 20 at the old Borders bookstore in Palo Alto. Moral of the story: it's important to play dumb where appropriate because most people are relative-intelligence insecure.
PS. III. Sorry, reader, for the rambling and discontinuous thoughts. Absurd endocrine values of unknown etiology currently... doctor appointments pending. ):
I have a sneaking suspicion that being "too smart" isn't why you bombed those interviews. I know I’d never want you on my team after reading just this one post, can’t imagine having to suffer through a multi-hour interview too.
> I bombed an Apple interview for a mid-career role by being too intelligent and too maverick compared to the group of compliant, I hate to say, yuppies. It was an interview panel of about 9 people and they were just speechless.
Are you sure they were speechless due to your high intelligence and maverick qualities?
> PS. I. Don't even get me started about a car full of CS IITians talking about the JEE and high-placers who seem normal on the way to snow country for snowboarding and gambling.
LinkedIn doesn't provide the vetting service Triplebyte does; they don't tell employers anything about job candidates' quality. It sounds like you want to pivot from the small market where you were the market leader into the larger market where LinkedIn is.
how do I reply to recruiter contacts? applied to a job, received a request for interview from company. Nowhere to reply on the platform? How do I get in contact with them?
Should there not be a reply button right there with recruiter's message?
p.s. I already emailed support but, my past experience tells me they won't reply.
What made me instantly distrust TB was the Publisher's Clearing House-style "you had the best score on our online assessment we've ever seen." C'mon, don't blow smoke up my ass right off-the-bat. If it's that easy, then it needs to be more difficult, broader, and deeper to pre-test candidates with proficiencies in particular areas with questions that are AI-generated: algorithms, data structures, particular languages, operating systems, networking, front-/back-end technologies and tools, software engineering, customer service, team communication common-sense, professional workplace behavior, and engineering ethics.
I went through triplebyte twice, once in late 2019 and then once again a few months later after the company I landed in shut down because of Covid.
I was happy with the process, they promised to take a lot of the headache regarding tech screening and it worked. I think the value proposition is clear - I don't want to do the same tech screens 100 times, and lots of smaller companies may not have the resources or desire to manage high quality tech screens themselves.
To be clear, I'm only addressing the comment about how people thinking highly of them are gullible. I had a great experience, twice, within a few months, and that included going through a time of huge uncertainty.
That being said, I was not happy to see each of the changes they've made over the last year or so, and this does not strike me a good direction for engineers. I suppose I have to agree that they are not doing this from a position of market leadership. Perhaps they were providing asymmetric value to engineers, and at the end of the day the engineers are not the ones paying for the service.
So, under the old model, order 200k engineers applied to us. Because we were exclusive gatekeepers, around 3% of those were "accepted" onto the platform. Around 2/3 of accepted candidates received an offer, and around 1/2 of offers were accepted.
I don't know much about them, but the post does say their goal was to be "exclusive gatekeepers". If you consider it from that perspective, they had more like a 1/3 conversion rate (2/3 * 1/2). But conversion kind of seems like the wrong metric.
What metric would make more sense here, though? I can agree that from TBs perspective, they may not factor total applicants as their base # (although that is clearly their market). However this exclusivity of only using the small percentage they accept doesn’t seem to be a scalable business model. Having 1% of the devs who try their service actually accept an offer would not entice me to try their service.
>However this exclusivity of only using the small percentage they accept doesn’t seem to be a scalable business model.
For sure. That's very likely why they're making this pivot. Exclusivity was their goal, but it just turned out to not be a very profitable goal for them, it seems. I was basically just saying it seemed to be a "theory" issue rather than an "implementation/execution" issue.
I noticed that too, and when you combine it with "Triplebyte has hundreds of thousands of engineers on our platform" it means they aren't doing great with the rate either.
> I forgot how to implement the zig operation in a splay tree.
employee at Triplebyte)
> To put some hard numbers to this: further down this thread, there's a post about how "any engineer" could answer a question the poster thinks is too easy. I looked up the question in our back end and, in fact, barely a third of people who take our quiz get it right (the correct answer isn't even the most common one!).
So there 1000 job matches and at most (("hundreds of thousands of engineers on our platform" / 3) - 1000) who were incentivized to answer these contrived problems, did so correctly and still couldn't be matched.
So even among the large pool of engineers who have gone thru the process, met some arbitrary threshold of engineer-ness, there's still a huge mismatch between corporate/prospective employee expectations, that I'm not sure will be able to be overcome quickly even with these new initiatives, but it's interesting that they are being pursed now (not surprisingly after the shift in working environment after massive government restrictions on freedom uber alles).
I agree. We tried using TripleByte for hiring but what they screen for and what matters are entirely different. A founder we knew got an angry missive from one of the TripleByte founders because they’d rejected candidates during a final culture screen. Apparently the only qualification should be whether the candidate can do contrived coding tests and programming jeopardy, but whether you actually want to work with them is beside the point!
> Apparently the only qualification should be whether the candidate can do contrived coding tests and programming jeopardy, but whether you actually want to work with them is beside the point!
Yeah, different companies want different things and are ok with different things. My best experiences have been with companies that have taken less than 2 weeks to hire me with no testing whatsoever: just a couple of calls. Other companies want and only select for the contrived coding tests and programming jeopardy, and that's fine, but I want nothing to do with them at all.
Maybe even allowing 3rd parties to bet on specific hiring decisions (in addition to what is described in the link) could add another layer of incentivized feedback loops to the process (esp if the people making the hire/fire before 3 months are made known) and add another layer of market driven signals for them to pay attentions to. Maybe make it more expensive to chase certain types that are in higher demand, and less expensive to chase others types that are in low demand (not the salary to the employee, just the payment for filtering). Maybe less expensive for shorter refund windows, and more expensive for longer refund windows.
When I went through it was like, ~1k/month and somewhere from 5k-15k/hire over the top. Industry average is around 10k/hire and their pricing suggests they're probably banking that amount with pricing used to encourage loyalty and appease investors with the Saas narrative. They're not pulling in much greater than industry average, and even if they're double, its only 20m, which is still bad.
Especially when they're probably making on average less than $25k per engineer they place. I can't imagine they're making even close to $100k on average.
$25k*10k = $25M / 10 years = $2.5M / year. That is peanuts compared to how many people they employ. Plus the ads they run...
I have no idea what Triplebyte gets or even what their business model is, but typical recruitment companies get a lot more than that -- on the order of 15-20% of the first year's salary, so on the order of $15-25k. I suspect Triplebyte is less than that, but could be an order of magnitude more than you've guessed.
There are two categories where people pivot while crushing it:
One. The problem domain they are in is on its way out, and they've decided to defect instead of being the ones left holding the bag.
Two. They hate what they do for a living.
The first company to stop making buggy whips probably did not do it because they weren't selling any. The fact that they ceded market share to their top competitors probably sunk the barb in deeper for them: Yeah people are buying cars but our sales numbers are still going up so why should we change?
In tech we see companies all the time end up being complicit in the destruction of their own industry by reducing its relevance in some manner or other, and by exiting early they have more options of destination. If I jump to the same industry as a competitor a year later I just look like a copycat. While some copycats copy other people, only better, they are the exception to the rule. Most churn out uninspiring derivative work, and if they can't at least do it cheaper then they end up on the scrapheap of history.
In our case, it's more just an intermediate state between "crushing it" and "failing". We had a product that was working for a subset of engineers, but was running into structural and scaling problems. Simply put, it was (sort of) "crushing it" but was foreseeably not going to keep doing so.
At that point, you can either keep going with what you're doing (and run into an inevitable wall later on) or you can foresee the future problems and pivot before they become unmanageable (which is what we did). Relatively few startups find their final model from minute one.
Funny but whenever I think of Triplebyte, all I remember is seeing their ads everywhere. I am personally a fan of Indeed and they do a good job sending leads and applicants other than good old Linkedin.
Based on my experience with them, I think they totally failed at their previous model.
When I was fresh out of university, I had trouble finding a job. I had worked part-time at a small company through the last few semesters, but that company blew up just before I graduated. Every interviewer I ever met with was impressed because I really knew my stuff, but had trouble getting in the door due to an unimpressive CV and awkward mannerisms.
I did very well at Triplebyte, scoring in their top bucket and getting top marks in several of the sub-categories. Those metrics are relative to all engineers, which would put me even higher amongst recent graduates. I also did very well on every software project I worked on and scored top marks on several other objective tests, like the Major Field Test.
I was exactly the sort of candidate their system should have been able to help. I never even got an email out of it, let alone an interview.
I remember my experience as a candidate on Hired. I pretty consistently got low-grade startup trash (think collectible art trading cards, but digital!) with lowballed comp. If Triplebyte was worse, then it's no wonder they placed so few people.
I’ve had a pretty good experience using Hired the past few months. After getting a lot of small startups, I put that I was only interested in larger companies with 500+ people and I don’t see as many now. There seem to be a fair amount of large companies hiring remote after the pandemic WFH experiment, which is great for me outside of a major tech hub. Most places meet or exceed my salary expectations. I’ve gotten interviews with a few companies I’m quite interested in and would have never thought to apply at. This has required relatively little effort on my part as I usually write a cover letter and tailor my resume only to get rejected without reason, but with Hired I just set up my profile and get hits. I definitely would recommend it to anyone looking around for another job.
100% this. They say individual engineers can't exercise this power, but that Triplebyte can use its weight to force their hand. As a hiring manager, I have dozens of recruiting platforms to choose from, most touting the same list of resumes. Yet another resume farm has zero power.
Fortunately, most of the dirt thrown at recruiting teams is wrong in the first place. I don't know any hiring managers who make lowball offers that have a low chance of acceptance - the costs of the process, the risks of hiring someone who starts mad at you, and the value of winning someone who is actually worth hiring, are all way more important than some incremental cash saved.
> As a hiring manager, I have dozens of recruiting platforms to choose from, most touting the same list of resumes. Yet another resume farm has zero power.
Well, presumably the idea is that engineers, especially high quality ones, will be drawn to the recruiting platform that has more pro-engineer policies. And so if you avoid using it, you lose access to that subset of engineers.
It's still a chicken and egg problem, though, because engineers have little incentive to join until there's enough companies.
Yeah, I think the biggest issue is with the implication that their existing user base, which was built under their old policies, is a unique asset. I'd be willing to bet that a significant majority of current TripleByte applicants also use other platforms.
However, I think that even if they build an attractive, differentiated experience, the advantage is unstable. The problem is that "traditionally strong" applicants (those who already have positive ROI from existing processes) are disadvantaged by being exclusive to one platform. So, from a hiring perspective, TripleByte's differentiation in terms of unique resumes is going to be from people who are harder to assess.
I'm actually fine with that as a value case - we've had great success hiring from bootcamps. "We help you reach strong nontraditional applicants" is a great value proposition. But it's not the same as a position of power to dictate the hiring process. Maybe a better pitch would be: we run a process that helps great non-traditional talent and great non-traditional companies find each other.
I'm really curious who is USING this company for their recruiting. As a IC I've been approached a lot by them, tried taking one of their tests, was turned off.
And as a HM, I was never interesting in using them, and we continue to use our own internal recruiters or local recruiters who know the market.
Having worked at a startup that was trying to disrupt the hiring/recruiting market back in 2010. I'm curious to how effective they were....if at all.
Edit: update, maybe I'm super skeptical because of all their cheesy ads that were all over Reddit for so long?
I interviewed with triplebyte about 5 years ago. They rejected me saying that my editor skills were good but I didn't pass the mark because I couldn't find enough bugs in a program fast enough while being watched. This was really good for me as my subsequent decisions took me to where I'm now as a lead architect of a US startup, earning 5 figures USD while living in LatAm.
On related matters. I know several companies that have pivoted out of the "hiring/recruiting" space because apparently that space is not easy to scale/automate.
I was pretty happy with triplebyte from the hiring side. Slogging through candidates with good looking resumes that couldn’t actually program was expensive for my employer and unpleasant for me. TB took care of that first big cut down.
I guess they didn’t hit product-market fit with candidates. Two sided markets are tough.
I think their process of 100% not working. They branded them self through advertising that they will provide constructive feedback if you don't crack their interviews so dev can improve upon shortcomings.
My experience was different I never got feedback after my last round part of the reason must be they were not able scale this process up.
I wouldn’t be so certain. Obviously, something isn’t working well or isn’t as good as it could be, but that doesn’t mean they are on the way out. Maybe they are, but plenty of businesses say, this isn’t working like we want it to-we can do better. As a worker, I am happy that this is a product that seems to empower workers over companies. I wish them all the luck and will look at their offering to see if it could benefit my own team.
I think the negative comments here are being a little unfair. It sounds like you've had a really terrible experience with TB. I get if that's your experience and interpretation, but TB are not superficial hawkers of products without fit or substance. They are passionate and intelligent and friendly engineers and weirdos who genuinely care about making a good startup and helping developers and companies match.
My experience is different. I've been through the TB process three times, back in 2017 where they flew me to SF from Asia for Flexport, Mixpanel and two smaller startups, and then again this year where I went to the final stage with FB production engineering and TB itself. And in between as well i created yet another profile on the system after shelving prior ones for a fresh start. I have a non traditional background, (ie, an "outsider" developer, chem degree and self taught hobbyist programmer since young), i take my craft seriously, have a bunch of popular open source products and a successful low touch indie business. But i don't consider future success guaranteed so i like to keep talking to companies to find that great opportunity for me. I know my tech skills in some areas are not the best so i like to stay in the loop, stay up to date and relevant and stay in touch with what companies are looking for. By far the best hiring process experiences I've had have all been through the TB platform. And my "open rate" for applications and inbound interest on TB is way higher than any other methods I've tried (SO jobs, WWR, HN). The quality of companies, hiring process and level of interest I've received via TB has been epic and, at least in the past, even the TB support staff are super helpful going so far as to provide personalized feedback on profile improvements and resumes. I'm not an expert at getting hired but i couldn't have asked for anything more from them. They were fantastic. I only have great and wonderful memories of the time i spent in hiring processes of companies i "met" via TB. So this is my experience of it, and this is despite the fact that i have never ever received an offer from any company I have met on TB, including from TB itself. I don't blame them. I just consider my own skills are not at the right level, and the fit is not right for the companies I've met, while i work to improve myself and my skills. That's all. It's a simple and beautiful experience I've had with them. Soooo different to hiring through other platforms (where funnily enough i have recieved offers). I've met and talked to ammon and many other TB team members and they have all been cool, friendly, passionate people. From what I'm reading of this pivot, they're being vulnerable and open to learning, and using what they've learned so far to create something even better. I'm not gullible, I'm not saying they will succeed. But they're trying. And they're earnest and sincere. What more can you ask for? This has been my deep experience with them. I think the most vocal critics are talking merely theoretically without any experience of TB nor deep nor accurate understanding of who they are, what they do and what they stand for. To them i say, give it a try. Investigate before contempt! ;P ;) xx
It's a brand of artificial lawn for sports arenas, plastic "grass", which has given rise to the idiom "astroturf campaign" = "fake grassroots campaign".
By far the most exciting thing about this announcement is this feature: "Detailed information on what a recruiter did with your application."
I've been wishing for something like this for years. Especially for early-career engineers or people from non-traditional backgrounds, this is insanely valuable because it helps you to know avoid wasting time on applications that will never go anywhere; avoid typing in your resume, line by line, into another form after already submitting your pdf. Avoid writing a cover letter to a job that's already been filled. Avoid applying to a posting that's really just out there for a company to "gauge" interest, not for filling a real role.
If an application is rejected, fine. But getting a follow-up request 7 months after submitting a resume into a black hole is ridiculous, and I think any system that decreases the information asymmetry between the applicant and the employer that allows people to intelligently approach their career search is going to be tapping into something truly valuable.
So fun fact, we built this feature as a pretty low-grade experiment, but when we ran it by actual users we got such a positive response that we ended up putting it at the top of our new front page.
Like you, we thought (correctly, as it turns out) that it would appeal to people early in their career. What surprised us is that very senior engineers also told us they loved it - we didn't think it was as much of a concern for them, but sometimes your users surprise you.
It's also a great example of how obvious low-hanging fruit gets missed until you start thinking strictly about "what can we do to make job seekers' lives better?". It's not a particularly innovative or difficult feature, and yet major job sites with a dozen times our engineering resources still haven't done it.
It's ridiculous. You can go through a couple screens, 5 coding interviews, and the end is just a "no" due to liability reasons. Literally zero information about why you didn't get an offer or if you were even close.
The last time I was a hiring manager, I committed to giving feedback to anyone I rejected after the very first screening step. This was about 25-30% of applicants, which (even for a role in considerably less demand than engineering jobs) ended up being many dozens of emails. It was a giant pain in the ass - and it's pretty emotionally draining because you're turning down people who would really, really benefit from getting hired. I did it anyway, because I'd promised to, but it was not fun.
It might not be fun but it is definitely good for your brand. In the last few years, I have interviewed at several dozen companies but the only companies which provided feedback to me were Facebook and Brex. This sub communicates to me that they take the candidate seriously and don't jerk them around.
Next time I look for a job, I will definitely be targeting both these companies.
Seriously. People ache for feedback in low information environments. Good on yall for taking customer surveys seriously (even if the data underlying it isn't always clear) because enthusiasm, though noisy, is a strong signal.
I wouldn't charge for the service, however, unless building it as a freemium model.
When I graduated I submitted around 50 applications to open jobs around where I live, to almost no feedback at all.
I'm in this industry and doing well because of recruiter reaching out to me (multiple times) and not vice versa. The whole idea of applying for jobs just doesn't seem to work at all.
My experience is similar but at a differet level of seniorty. I've applied to ~15 jobs for engineering manager and got 1 answer.
On the other end a recruiter reached out to me and I'll be doing the final interview with their VP next week.
I don't think we can draw conclusion at that stage (we are a dataset of 2..). But I'd love to have access to the LinkedIn dataset to figure out if applying for jobs is broken at scale.
Waiting for a recuitor to call you limits your potential but makes the process easier for you. Applying yourself takes a lot of work but you get to select who you want and to target a bigger group which. But it's messy, ugly , stressful and filled with rejections for no reason.
Doing both, when you need a job, gives you the best of both worlds.
It's ugly and mimicks real life... more effort gives better results but life isn't fair and never has been. Luck factors but always fight on multiple fronts.
Then I stopped bothering when it's super obvious that isn't working. Particularly, on Linkedin you could see the number of people that has applied (maybe a paid feature, i don't remember now). And for any entry level software engineering and data science position it's always in the 100s.
The one recruiter that I had contact with did more work than all of these. I've since switched jobs (to one of the FAANG) and that was through recruiter too.
A benefit of Triplebyte to big enterprises not mentioned in Triplebyte’s discussion here: acting as engineering assessment proxy for engineering hiring managers stuck with pathologically risk averse enterprise HR departments.
Often enterprise HR is paranoid of honestly evaluating anyone for anything, to the point that many HR teams tell one another and engineering hiring managers that it’s “illegal” to assess candidate abilities on the way in the door, at all.
If you think about joining a dev team where no one checked if any candidate could FizzBuzz, you can imagine the workplace environment that can end up with.
Triplebyte is able to provide engineering managers with a stack of pre-vetted resumes so an enterprise can interview by its lowest common denominator HR policies, while still having a prayer a team will at least be made of candidates who can code.
The value of this to an enterprise stuck in this position is hard to overstate.
Hopefully along with expanding the talent pool per this post, Triplebyte can figure out a way to get well paid for this ability to help land actual coders on actual dev teams despite enterprise HR.
The need for this is huge.
// I see a comment below from a Triplebyte PM about score matching: https://news.ycombinator.com/item?id=27542621 … That would be slick: land candidates that genuinely raise the bar, but aren’t so far ahead as to be speaking a foreign language to the team. Of course, this would require also assessing the target team, and oops, we’re right back up against that HR policy…
There's an old quote: "Nobody ever got fired for buying IBM."
Large companies tend to be more risk-averse, so if there are two companies you can use for a service — $PromisingStartup and IBM — large companies will generally make a decision that's optimal to the specific decision-maker, not optimal to the company.
There are two options if you, as the buyer, choose either $PromisingStartup or IBM: success or failure.
If IBM or the $PromisingStartup succeed, then you've done your job.
If IBM fails, you can tell your manager "Who could've guessed! It's IBM!"
If $PromisingStartup fails, you'll have a harder time explaining your decision, and the fault will be with you.
The "Nobody ever got fired for choosing IBM" idea is useful to look at every decision large companies make, whether it's pivoting, choosing a SaaS product, or hiring.
TripleByte, in its current form, has been beneficial to candidates as well by giving enterprise employees a justifiable signal towards hiring them regardless of pedigree.
> If you think about joining a dev team where no one checked if any candidate could FizzBuzz, you can imagine the workplace environment that can end up with.
That sounds hilarious How could a dev team not be able to write 5 lines?
You would be surprised how many people legitimately can't wrap their heads around a fizz buzz style problem, but also how many people who can yet completely freeze in an interview setting.
> Performing an interview is a different skill from writing code.
I think this is true for the day-long leetcode interviews, but to succeed as an engineer, even at the entry level, I think you need to be able to talk through how you would solve fizzbuzz. A big part of the job is communicating trade offs with other engineers.
You're not wrong, but the problem is that interviews aren't actually about jobs. The average person interviews because they need the money. So from the start they're not thinking in the programming space. The pressure for them is to make the interviewer believe they are worthy of the job.
I think if I could describe my personal experiences, I'd say that on a subconscious level, I know that how I speak and articulate details will be observed by the interviewer through a cultural lens more often than not. This may be unique to my personal background, so maybe this is different for other groups. But interviewers often get hung up on the use of different language/terminology, the types of jokes/jests that enter the conversation, how I respond to their language. And there's sometimes been an unwillingness to allow different terminology, perhaps because that's all the interviewer themselves knows and understands.
All that to say, a code interview doesn't always come down to the quality of the code.
I got shitlisted at one place for being the only dev with the mathematical insight to know that multiplying pesos by a dollars:yen ratio doesn't yield a correct conversion of pesos to yen. They let that one slide without fixing it, but the final straw that got me frogmarched was refactoring an if-else to a switch.
It is hilarious but man, you would be really surprised how people choke on this. It's weird. People with 10+ years writing software can't do this sometimes.
I was once accused of lying about my work experience by a $FORTUNE500 interviewer when I failed to articulate what a CSS flexbox was for a backend dev position.
Such an eye-opening experience! Now that I’m on the other side of the table these days, I understand the importance of making interviewees feel comfortable and asking questions that allow them to start off strong.
Last year I did a project for an org that has a similarly sized team of developers. They could not figure out html and css. Hadn’t heard about git. And worse is that they developed on the production server. I’m not making this shit up.
They saw me as a god for fixing their shit and offered to hire me. No thanks!
My random Triplebyte anecdote is that I did good on the quizzes and gave a solid interview, but they declined to represent me. They were transparent about their reasoning: they saw me as somewhat of a specialist in parsing, and they didn't know how to place someone with that peculiar skillset.
Which turned out fine. I found a job where my pre-computer programming work experience was relevant, and here a couple jobs later I'm working heavily with and on parsers.
So. Count me as one engineer they rejected who doesn't hold that against them.
I'm pessimistic about this pivot though, for a couple reasons. One, it smells desperate. Recruiting is sales, and in sales, desperation is the kiss of death. I can't read that blog post without coming away with the strong sense that if this doesn't work, it's over for them. If I can see it, anyone can see it.
Two, companies aren't going to want to work with a firm that's allowing engineers to collect detailed information on douchey behaviour that the company might engage in. Does Triplebyte have the leverage and moxie to make them engage anyway? Probably not, see point one.
I wish them well, because recruiting is awful, interviewing is broken, and engineers deserve to have a better time of it given how demand-driven the market is, and likely will remain for quite some time.
Can't help thinking they'll be writing about their incredible journey within a year or two.
> Two, companies aren't going to want to work with a firm that's allowing engineers to collect detailed information on douchey behaviour that the company might engage in.
On the other hand, if a company does not engage in douchey behaviour, then using Triplebyte could be a way to signal that.
This fails a bit if companies can be credibly accused of douchey behaviour when they haven't been engaged in it. I don't know what Triplebyte's plan is here.
> Does Triplebyte have the leverage and moxie to make them engage anyway? Probably not, see point one.
Indeed. They labour the point that programmers have the upper hand in hiring, but that doesn't translate to Triplebyte having the upper hand!
Same thinking here. Can’t bite the hand that feeds. The thing is you have to chose a camp these days, it seems. Either the corp or the devs. If choosing devs, i think the biz model would work if it would be a meritocratic limited partnership that actually owns the new TB. As such the interviewer has skin in the game. Maybe like an Order? Or a League? The mission statement is not a match right now.
What companies want from us, more than anything else, is volume. They want to find as many engineers as possible and make hires. That's the thrust of the economic argument we're trying to make here: our incentives even with companies as our paying clients are to create an engineer-friendly platform.
I'm personally very excited to see this change by an industry leader! I think that there's a seismic shift waiting to happen w.r.t. engineer hiring. The "old guard" of hazing, quizzing, and gotcha-style interviews are slowly losing ground.
The most friction when switching jobs is the interview process -- and the question is why? I've been writing code for a decade+ now, working on startups, for known tech companies, public open-source, and have written a freakin' book! But, oh, my bad, I couldn't figure out a solution to your optimization question. I forgot how to implement the zig operation in a splay tree. I might simply not function well under pressure. Maybe I'm having a bad day.
Companies are missing out on literal geniuses by using outdated hiring practices. And, I get it, Google doesn't care. Amazon doesn't care. (There's an argument that they should.) They're huge and get X,XXX applicants daily. But why are small startups using the same hiring methodologies? It quite literally makes no sense. They're shooting themselves in the foot. I'm very passionate about this, and I'm working on a book on how to hire engineers. Maybe I'll actually finish it one of these days :)
For what it's worth on the hiring side, applicants lie (or more charitably, wildly overestimate their own abilities) all the time. Of resumes I see, 50% of them will not show up on time to their interview. Of the 50% that do, 80% cannot pass fizzbuzz.
Past companies and open source contributions are not perfect indicators. I've seen people with stellar resumes fail to even know the syntax for an 'if' statement in any language of their choosing. I've had people absolutely ace takehome exams, yet in person don't know how to write a function.
False positives are also much more expensive than false negatives, so companies would rather accidentally weed out good candidates than risk hiring bad ones.
If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
> Past companies and open source contributions are not perfect indicators.
This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
> If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
I've seen similar. I wouldn't say 80%, but somewhere in the 20% range of people on phone screens. If I ask a similar modulo question that isn't obviously a rephrased FizzBuzz (i.e. "write a function that returns every 4th item in a list"), that percentage goes up dramatically. A lot of candidates memorize FizzBuzz without actually understanding what's happening and how to use it.
> This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
There are no time constraints, and no way to tell how much of their contribution is from their own knowledge and how much is copied from StackOverflow/other open source projects/etc. The point of view is consistent; I can't run a marathon, but I can walk 26 miles. That's fine for the local charity marathon, but it's not going to cut it if I want to do competitive marathons. The constraints and expectations are wildly different.
Programming verbally over the phone is a skill that requires learning.
I can run a marathon but I can't run 100. Your interview is one of many that I am making time over lunch. I'm tired and getting a perfect mark on a take home isn't as important as getting it done quickly so I can find time to apply to other positions before I go back to work.
If you are not paying someone to do the interview you are not going to get anyone's best. Even if you pay people are trying to juggle things to get the interview to work.
And to the parent post. 50% of people being late means they took the time out to talk to you and come to your offices and didn't gage the time or didn't think it mattered. That doesn't make them bad programmers and shouldn't even matter to the interviewer. Was the car service the employer used late picking the employee or did they have to come on their own?
Even ignoring the fact that you keep moving the goalposts and are taking my "marathon" analogy way too literally, the argument that any kind of real-world coding involves "time constraints" akin to a 30-minute stress-ridden whiteboard interview is just bonkers.
>This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase?
They could maybe just be making small or simple changes. Or perhaps mostly working on markup, templates, CSS, etc.
It's possible they know a bit of programming but just don't happen to know how to check if a number is evenly divisible by another number. This isn't something an open source project would necessarily know you don't know. (I agree it's something any programmer should know, but I can see how someone could contribute certain things to certain projects without that knowledge gap being discovered.)
I learned about the modulo operator very early on, but if I didn't know it existed, I'd admittedly struggle with the question for a bit and would seem like I don't know how to program; especially in a pressured interview environment. If it were Python or something I think I'd come up with something awkward like `if i / divisor == float(int(i / divisor))` within a minute or a few minutes, but who knows, maybe the pressure would mess with me.
> If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
I dunno about the back half (blowing up million dollar system) but the front half, certainly.
We used to ask "write min()" at a previous employ, and the pass rate was probably <50%. I still to this day structure my coding question around "it has to get them to write a for() loop". It's not trick questions (I understand why people don't like those, and those can die out, yes.) it's things like "here's a sample grammar, how would we go about parsing it?" which is something that I've used any number of times in my career.¹ Other candidates fail to display understanding, let alone deep understanding, about things like HTTP or Linux. Not exactly niche topics, and during this example period my employer at the time was clearly posting for "backend software engineer". And not just one or the other, but all of them, simultaneously. Since I can't establish any proof of anything, so… what other choice is there, but to pass?
¹while there are parsing libraries, yes, and I'll use those first / when I can, I would also say that 50% of the time, for whatever reason — better error reporting, better control — I end up going with a partially or mostly hand-rolled state machine or recursive descent parser. Also, while I wish people would just encode their data in something like JSON, or YAML, or whatever, it doesn't really matter, they. keep. making. new. grammars. Often inside those other grammars. Just today I had to deal with a config — a YAML config! — that wanted "key values". Turns out, it wanted strings formatted like "key:value", not YAML mappings.
> If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection.
With a bad pipeline, it's extremely likely, especially if you cheap out and try the bootcamp route (there's a reason Lambda is desperate enough they will "loan" a student you can try before you hire!).
To put some hard numbers to this: further down this thread, there's a post about how "any engineer" could answer a question the poster thinks is too easy. I looked up the question in our back end and, in fact, barely a third of people who take our quiz get it right (the correct answer isn't even the most common one!).
That's maybe a little unfair (plenty of people who aren't engineers take our quiz out of curiosity), but the basic point here - that companies are frustrated because they keep getting fizzbuzz-incapable applicants - is more-or-less accurate. That has been (and continues to be) a big part of our offering on the other side of our platform: in the same way that we can provide data about company behavior to engineers, we can provide independent verification of an applicant's skills to companies.
> Of resumes I see, 50% of them will not show up on time to their interview. Of the 50% that do, 80% cannot pass fizzbuzz.
Wow, sounds like you either need to change your sourcing or improve your early screening. If that many poor candidates are making it to the interview stage something is wrong with the process.
I should have clarified, the first interview is just a code screen. People submit resumes, and we test them with fizzbuzz. 90% of applicants are removed at this first filter.
> And, I get it, Google doesn't care. Amazon doesn't care. (There's an argument that they should.)
And yet they will still email you, year after year… as a dev that's worked in finance/academic research labs and for low funded/bootstraped startups all over the world for the past 10 years… I'll never get into these big companies because I simply try to find companies that need work done yesterday, see my experience (maybe reach out to past corps i've done work for), and give me an offer for at least a short term contract (and these big companies will never do this).
The worst is when I've think I found an interesting company (that on more than one occasion, publicly likes to complain about not having enough devs… lol), start talking a bit and they send me a triplebyte link… maybe it will be different this time, but I doubt it because these behaviors are too ingrained in a lot of corporate processes (even if its mostly "big co does this, me small co must copy"-NPC type thinking)…
I agree, To me discovery process is broken. FANG sends email every year once you have already reached onsite. But I am trying to get into this tech companies new IPO's not FANG. I have applied to many good ones.
I have Github repo.
I have my own portfolio.
I have wrote flutter app deployed to play store to demonstrate I can function individually and still can write production ready code.
And All the above just on my own spare time.
But every company I have applied told me my skill set doesn't aligned with them.
I get response from them that there is no traction on my resume. While some friends get interview after interview and others don't.
I am not saying that I am the best but we need different process where entry is given and now show me what you can do.
What I find particularly jarring about tech interviews is that they often don't adapt to the actual role you're interviewing for.
I moved from engineering to management over 5 years ago now and still in interviews for positions like "head of" in 100+ people companies or other management roles in bigger companies I got asked the usual algorithm questions.
I just have zero interest to prepare for these interviews, I don't have the time or the motivation, and I honestly just don't care about the answers. Most of my job, and my team's job, has nothing to do with them.
The world might be different in FAANG, and admittedly I have not worked there (though I have worked for other top companies) – but most companies are not at that scale, and they should not use the same metrics, particularly for management roles.
Decades of poor technical interview practices started by a blog post (the one about fizzbuzz). It's amazing how a single blog post changed the entire industry by causing such fear about "false positives". Looking at that blog post now, it's obvious how it lacks real analysis of the situation:
* The blog post complains that even experienced engineers they interviewed couldn't write some simple code on paper. Did anyone stop and think that maybe it's writing code on paper in an interview setting that's the problem? Stop and think about this. Imagine writing code like you usually do in a text editor. And then writing code with pen and paper. Backspace? Can't do that. Copy/paste some line to move code? Can't do that. Add some newlines? Can't do that. Run the code using the compiler? Can't do that. Of course most developers, even experienced ones, will do poorly under these conditions, this is not how code is normally written. On top of the already stressful situation of an interview, there's this suddenly completely foreign and limited way to write code.
* Even the blog post only mentioned fizzbuzz as a quick, easy question to filter out zero coding ability. Another way to say this, it was meant to filter out coding skill <= 0. Not skill <= 50 or skill <= 90. Zero. After that, the interview was supposed to move on to things that actually matter for software engineering. The present day interview has strayed extremely far from this. These days, you get asked two leetcode questions during an interview and even if you perfectly answer the first one, if you don't get the second one then you're an automatic fail. That was certainly not the intent of the original fizzbuzz, since answering any leetcode clearly shows coding ability. And the interview never has a chance to move on to the part that actually discusses software engineering skills. Or even if it does and they do very well on those interviews, the failure of only getting 1/2 leetcode questions correct overrules everything else since they must avoid "false positives".
(N.B. I was being a bit hyperbolic, but, yes, for example, I highly doubt Steve Jobs could.) The spirit of my point is that most smart people don't care about the kind of idiotic minutia typical hiring panels ask. They care about interesting/creative problems.
I mean Steve Jobs may not be a great software developer. I don't think companies are hiring for a Steve Jobs. You could be really good at other things and not be great at programming
A SJ type is what TB would need to orient this pivot… to turn the problem around. Sourcing ideas on HN will only lead to “faster horses” suggestions. BTW did the TB interview after been lured by dark patterns on social media a while back when I was still using. Got the bloom filter from a hipster that was playing on his phone while I coded tictactoe in a langage I hadn’t used In years. Complete waste of time. Then they wanted to make profiles I hadn’t signed up for public, good thing they listened to the backlash. Just saying; it seems the management needs to use an hypothetical service like theirs but that works to find someone to help them on the angle for that pivot… a headhunter ?:-)
Tongue in cheek, good luck on the pivot guys, it is a tough problem.
10 years ago, I built chooseyourboss.com from scratch for a client of mine (with one of my best friends).
Our principles at the time were very similar to what Triplebyte is trying to do now: give power back to developers.
We build a version of "hired" (did they die?) before "hired" even exist. A matching platform that was matching people with companies that was sharing both their desires and values. Small/big company. Consulting/Product company. How much $$ do you want? What technologies you're interested in? Where do you want to work? All of this anonymously of course.
It was all fine and dandy, the company actually still exists and generates revenue, but I'm overall very bearish on the recruiting space.
It's a sales game. It's a numbers game. And on top of this the people top companies REALLY want to hire aren't hanging out in these recruiting platforms. So as a platform you're only feeding average/average low candidates to companies, and in the meantime, companies that have in house or externalized sourcing resources get better results by going over linkedin and their employees network.
My problem with these platforms is that I never end up ‘matching’ with any companies because almost without exception their ethos is ‘cheap and fast’. The only way to change that is from the inside.
Very insightful comment that I wish was higher up. The devs complaining here don’t know how tough a nut to crack it is to build a truly applicant-centric platform profitably.
I'd love to work with Triplebyte (as someone who's running a hiring process), but a while back they removed their option to work on contingency, and it's a difficult sell to fork over $X,000k for another source of engineers when it would just be one of 10 sources we already work with.
Maybe I'm wrong here, but it gives me the impression that Triplebyte doesn't believe in it's ability to get candidates into jobs. If companies pay (a not small sum) when candidates actually get hired, then I feel that incentives are aligned. If it's an annual fee (with no option for contingency) then I wonder what incentive the company has to make sure what I care about (making a hire) is the thing they care about.
90% of the other major players (AngelList, Hired, etc) in the market offer this... so why not Triplebyte? Perhaps I'm missing something.
The main problem we had with the contingency model is that it resulted in companies using us for a few of their "hardest" hires, rather than as a bread-and-butter part of their recruiting. For example, almost no one wants to pay a contingency fee on junior or remote hires (because they are perceived as 'easier'). This made our platform kind of suck for junior and remote engineers. The subscription model is more of a commitment from a company (I understand why some companies don't choose to use us.) But it means that once a company signs up, they want to make as many hires as possible through us.
What about a sliding scale for the commission? The higher the salary the bigger it is? So a junior engineer would practically be free, but a senior engineer would bring in more revenue than the subscription.
Then the company is incentivized to use you for junior hires. Maybe you'll get a few freeloaders who only want to hire juniors, but even those will eventually be seniors who love your platform, and at the same time, will boost your stats on the number of juniors you got jobs for.
Interesting... you've probably considered this but, sounds like to solve that problem you could just have an option for companies to subscribe, or be on contingency? Then the companies hiring for juniors can subscribe and the folks looking for harder hires can be on contingency?
"But it means that once a company signs up, they want to make as many hires as possible through us.", right, but as a hiring manager that doesn't make sense to me... I don't care that we're making hires to justify where we spend our sourcing money. I care that we're making great hires, so I want to cast as wide a net as possible. I'm not going to shut my eyes to other sources because I've paid for triplebyte. But maybe in bigger companies I wouldn't be the one making purchasing decisions? And the folks making those decisions would be pushing HMs to use Triplebyte?
It is interesting, how the incentives align and don't align sometimes. Hard to say if I'm even in the target market (raised $15m, hiring a few folks this year in the "hardest" category, #1 problem is qualified folks at top-o-funnel).
Maybe I also have a personal penchant for seeing companies who bet on themselves and clearly align incentives :shrug:
That business model is optimising for large customers that are probably better equipped to replace you with something in house. And leaving the long tail of smaller customers on the table.
I wanted to use triplebyte - but I only want to make 3 hires per year.
Sounds like you hit a niche and solved hard problems for companies. Unfortunately not a niche big enough for a company that's taken 10+ million in VC money.
We experimented with this, as well as with trial periods (which are sort of the same thing, but with a time limit). Trial periods ended up being easier to think about and optimize (each account either converts or not at the end of the period). But we may revisit this in the future.
Would be interesting if you could do something like:
-You vet engineers with however you/or individuals engineers see fit (test/ past work/ opensource / interviews/ etc), they highlight what they want + maybe what your data says would help them stand out for the "type" of dev they are.
- force companies to pay you $xk - $1xk to hire them right away if they "match" with someone they select, but with the ability to get a refund if they fire them within 3 months (but like 60 business days to get a refund, and allow them to keep picking but reset the window if they match with someone else again).
- keep in touch with those employees during that window
That way, companies will eventually get their money if they make a bad hire and still be willing to use you again, you can collect more data on the larger part of the employment life cycle and basically get 60 day interest free loans at worst.
Why not just offer junior engineers for free? If companies don't want to pay you for junior engineers, they're not going to suddenly start caring when you're giving them 10 instead of 1. If they're paying you for seniors just let them pay you for seniors.
For junior talent I found it's much better to simply fork the $X,000k to a serious school with a good engineering program and run hiring events or sponsor things. Or establish a relationship with a research lab in the school that's close to your area of expertise.
As always with posts about employment, the idea that there's higher demand than supply for engineers is a lie. The demand for highly experienced cheap engineers is high, the demand for junior developers is almost non existent. New CS grads struggle to find employment, there are tens of thousands of very capable devs in open source that cannot find employment. Nobody hears about it because only those that succeed are actually spending time on forums like these talking about it.
We have a big problem in the culture of employing developers that is closely related to how the startup market operates. The market is willing to shovel money at a problem rather than spend time, therefore the market will pay someone with specific experience in their problem far more than pay someone less and have more time to learn the domain. This results in siphoning of experienced developers away from new developers, which artificially reduces the number of experienced devs. The market is creating the problem for short term profit, and then turning around, lying and saying "there's not enough devs".
> As always with posts about employment, the idea that there's higher demand than supply for engineers is a lie. The demand for highly experienced cheap engineers is high, the demand for junior developers is almost non existent.
Yeah I don’t think this is true. I teach undergraduate CS and our data shows (which we publish at admissions so is public) we regularly have close to 100% job placement within 6 months, with 90+% of that being CS related. I know many of the students personally and they are almost all landing good software development jobs. I think we have a good program but I hear the same from colleagues at other schools.
“Good software development jobs” of course is a relative term. CS related does not mean Amazon at >$100k, although a few achieve that.
I've seen now multiple mail in letters that ask "Have you found employment related to your degree", or a question similiar to it. If you have a compsci degree and end up working an IT phone support job that is counted as "related". Which completely skews the results. Or, better yet, some ask "are you gainfully employed?".
There's good programs out there, but they are in the minority. If you run a good program, that's great. In my experience most places offering compsci are there to take the federal loan money and hand you a piece of paper.
This is true, at least of developers who lack both experience and impressive credentials. A new grad from, say, UC Berkeley might be fine, a new grad from my own alma mater (which has a direction in its name) will often struggle, and a hobbyist without a CS degree has an even bigger uphill battle.
The reason we didn't talk about it as much in the main post is that our approach to the problem of credentials hasn't changed all that much. We still have quizzes, and someone who is having a hard time getting their foot in the door (a) is likely to be willing to put in the effort to take them and (b) will benefit from doing so. That's not a complete solution to the problem (it helps people with skills get hired, but doesn't in itself help people get skills that you only get on the job), but it's better than the status quo.
We haven't forgotten about people struggling as they look for their first job. (And I certainly never will: to get personal for a sec, my first job search was a miserable multi-year affair that nearly killed me.) We've got some experimental irons in the fire here (e.g. a program that basically highlights people who haven't gotten as much attention as they 'should' based on their skills) that we're not quite ready to talk about in great detail. If they work, you'll hear more about them soon. If they don't, we'll try something else.
> hobbyist without a CS degree has an even bigger uphill battle
This was me when I tried Triplebyte. I wish it had worked out. I don't really blame TB for the failure, but I think there was a lot of optimism on the part of TB that wasn't shared by the companies I interviewed with. TB was just another placement option for every one of them, and I found it challenging to overcome what felt like walking into several onsite interviews completely blind to what was expected of me.
Sorry, but I call major bullshit on this. Sure, if you have a D average at a low tier school, perhaps. But if you are a competent coder who got good grades in school, I see many companies ready to hire new grads left and right (obviously folks who graduated during the height of the pandemic had different issues to deal with). I don't know if we're quite at the "anyone who can fog a mirror can get a job" stage like we were in the original dot com boom, but we're not that far away.
I know 3 of them. They only receive responses from jobs similiar to, but not actually software dev. Basically tech support jobs but you have to know the basics of a windows/bash command line. No dev work involved.
Companies advertise they're hiring. Like they advertise that they're profitable, make the best products, and are better than their competition. That doesn't mean it's true. Many positions are "nice to have", so they will stay there until they get someone with far more experience on paper than they actually need - someone willing to get paid less of course.
People lie, especially companies. That's the nature of the market. Even if we had 5x the devs than employers, employers would still say otherwise. This is because more competition between applicants means you can pay people less.
Infact even during the dotcom boom my sister actually got a degree in "web development", and nobody wanted anything to do with her. I don't know who comes out with these lies about "high demand" employment but so far all I've seen is confirmation bias by those already employed - ignoring all the rest of the people who can't get anywhere, leave it behind, and go work at something else.
The data doesn't bear this out though -- the median* salary for a software developer is not only substantially higher than the national average salary, the rate of growth is also higher:
That's empirical evidence of a tight labor market that's difficult to fake. Why else would salaries rise so quickly from such a high base? Even if employers lie (they do), the salary numbers from BLS are separate from that.
Of course, there are always people who slip through the cracks unfortunately. But avoiding anecdotes and looking purely at the numbers, it's difficult to find a labor market in the USA that's tighter than the one for software engineers.
I doubt people only wanting to move to Seattle/SF/NYC is the problem. What about all the people who only look for jobs in their hometown or regional hub where there are approximately 0 jobs for software engineers? From what I’ve seen that is a much more common problem, people basically limiting their search parameters to a very small set of all open jobs, rather than looking at the three cities that have the most jobs.
This, for sure. And you definitely don’t need to go to SF or Seattle, there’s plenty of jobs in cities like Chicago, Austin, Salt Lake City, Raleigh, Atlanta, etc. But there may be be much less options in rust belt towns and rural areas.
That was a bit of the "Reluctance to move" aspect.
I've moved across country twice and from one end of a state to another twice... its a pain, but I like new places.
One of the companies that I worked for (smallish city in rural midwest). There were a few tech jobs there (and a reasonable sized tech company)... several of my co-workers had never been further than about 100-200 miles away from where they were born (and then, only as vacation).
Yea... the reluctance to move is a problem.
I attribute part of this to my (gen X) and the "this is how things are" that I got from my parents (boomers).
My father's family had a few moves through the years... not too many, but a few. My mother's family moved several times (and once even to another territory) as part of my grandfather's career. Once my parents were married, they moved twice before settling down.
For me, when I graduated college, it was almost expected that I move somewhere else for a time. And I did. My brother too... and my sister. My brother (gen X) has moved five times since graduating college and my sister (gen X / millennial cusp) has also had five major moves since graduating college.
I believe there's a generational shift in that expectation of moving out - possibly from the financial insecurities that the dot com crash, '09 recession, and the '20 recession have instilled in the millennials and gen Z as they look for jobs... though that's just a guess.
> rather than looking at the three cities that have the most jobs.
There are a lot of cities/metro areas that have a lot of jobs that people tend to pass by. https://www.bls.gov/oes/current/oes151251.htm - note that there are other classifications that are applicable, but I can't get a page that has a set of them together... so just using this one as an example.
If you look at the raw employment figures, Dallas, Chicago, DC, and LA are way up there too.
If you expand this to the location quotient (based on how many jobs there are compared too other jobs) you'll find many other cities have quite a few jobs too. Huntsville, Durham, and Madison are also up there.
I'm an engineer who was introduced to a startup via triplebyte. I liked the platform because I went through a technical interview to even be allowed to connect with companies (one that a lot of my peers who are junior level weren't able to pass.) I guess my first thought is that doesn't this defeat the initial purpose of Triplebyte?
If you allow everyone in, regardless of if they are qualified to be a senior level engineer, will this not just be another job hunt platform where (when I look for another job) I'd still have to do another technical interview at the company itself?
How will companies know now who has been vetted as qualified and who is just lying on their resume? Maybe it's shitty to think this way but there is already a ton of sites out there that allow me to send my resume (or allow a company to view mine) before starting a traditional hiring process that Triplebyte prevented, allowing companies to connect with qualified high level engineers and know the person would fit their skill needs. The way it was companies mainly were just testing for cultural fits, etc.
(I'm a PM at Triplebyte working on this overall direction, joining Ammon for this thread.)
To be clear, we still have quizzes and the ability to show your performance on them to companies. The fact that they’re not completely mandatory to use Triplebyte doesn’t mean that they’re not important or that we won’t build further features around them.
Right now, companies can offer an expedited process as part of their reach-out. We’re working on ways to productize this a little bit better. We didn’t announce the feature in this post because it isn’t done yet, but one thing we’re looking at is allowing companies to set individual score thresholds for various types of expedited process (e.g. “if you scored a 3 on algos and a 4 on back-end, we’ll definitely get on the phone with you” or “if you have a 5 on python we’ll skip tech screens for you”). We certainly recognize that FastTrack was valuable to a subset of engineers, and we want to recreate the value it offered in a way that is a little bit more sensitive to the specific company and specific engineer in question. We imagine, for example, that less-prestigious companies (who are more concerned about attracting applicants) will probably set lower thresholds than the Apples of the world. Under the old system, we had to set a single threshold, which would necessarily either be too low for prestige companies or too high for everybody else.
And just to lean a little bit back into the pitch we’re trying to make here: if you’re concerned about interviewing process, wouldn’t it be nice to be able to search companies by how their interview process works? That’s not an axis on which companies meaningfully compete right now, but with the right incentives, companies will cut a lot of the annoying hassle that they currently have no reason to get rid of.
One thing I didn't like about TB is that I couldn't see my own score, or how I compared to others. They sent me follow-up emails, tried to engage me in interviews, asked if I had any questions, and when I asked about my score or how it related to others (top N% for example), they didn't even respond.
If a company asks me if I have questions, and I ask them, and they ignore me, I'm done.
I wasn't looking for a job, but thought the test results would be interesting. Who takes a test and doesn't get results?
This must have been a long time ago! We've told you what your scores were and how they compared for quite a long time now. Even the current version of that page, which is like the third or fourth iteration, is now many months old. (That said, we've got a bug breaking that display right now. Working on it!)
Speaking of your tests - your general coding skills one has an actual bug in the code, as of a few hours ago (I sent an email about it, hopefully it gets fixed)
I'll broadly second this. As an experienced dev, what kept Triplebyte on my radar was being able to at least theoretically cut past the initial round of 'can you actually fizzbuzz or are you just lying about it' that most companies need to filter out the people with inexplicably good resumes but no practical skills.
I think my experience sounds like what the article is trying to say…
I tried TripleByte and wanted to like it, but mostly it didn’t seem like anything special. The whole concept of getting pre-screened, getting multiple offers, and jumping straight to final interviews just didn’t pan out. The last company I interviewed with basically treated my application like any other pipeline. I had a standard recruiter call, hiring manager call, and several technical interviews before getting an offer. And the experience with other companies was similar. From the outside looking in, it appeared that there was no special track for TripleByte applicants. In the end it felt like the TripleByte process simply made the interview process longer, not shorter. Eventually I found it faster & more effective to pursue companies on my own.
TL;DR — companies I met through TripleByte ran me through the standard process I’d experience through any third party recruiter without the personal touch one might get from a good third party recruiter.
This wasn’t my experience. Sure they didn’t just make an offer but instead of the usual algorithm test and a second interview my onsites consisted of actually working alongside the founder for a day and then an offer a few days later. I’m totally fine with that what I don’t want is to have to do 2-3 interviews at every company I apply to.
Triplebyte was basically friction. It didn't help you skip the onsite and instead forced you to do two onsites, once with Triplebyte and once with the hiring company. It didn't make sense to ever engage with Triplebyte.
It should be an upgrade to do an onsite with triplebyte and an onsite with each company instead of doing several phone screens with each company and an onsite with each company.
Exclusivity and rapid growth don't normally go hand-in-hand, so one would expect that at some point, the company would have to bend on one of them, and it makes sense that they decided to bend on exclusivity. Facebook followed a similar course early in its history, and what allowed it to rapidly grow was diversifying its value prop. When it was exclusive to Ivy Leaguers, the draw was the meet other people at your school. When it was exclusive to college students, the draw was to meet other people at your school. But then when it became open to the world, it became much more of a general-purpose social networking site. That's the course Triplebyte will likely need to navigate if it wants to be successful in the long run.
The problem with Triplebyte IMHO is that what the originally promised, while a great idea and concept, it cannot scale. Also after interviewing with them for a role in their company I got to realize they don't know how to conduct interviews themselves.
I used them two times. The first one was very early on, where I was given a home assignment and interviewed on it by Aaron himself, if I am not mistaken. The dude was awesome at interviewing, and knew exactly how to probe to get a better understanding of your skills.
That was when they were promising that you can interview with them so you don't have to do technical interviews with the companies. I thought it was an awesome concept and could really reshape hiring in the tech industry.
Second time was a couple of years ago, where the model has already changed a bit. Passed the first round and one of the companies that I could interview for was triplebyte themselves.
What a disappointment! The only difference in the interview process than the rest of the companies was that they gave you a laptop and asked you to do practical coding instead of whiteboard generic algorithm solving.
Some of the interviewers themselves were junior members of the stuff with 0 experience in interviewing.
Scheduled a practice interview and the only slot available was ~6AM my time. Nobody showed up and I wasn't informed that they wouldn't be showing up.
After reaching out about it -- they tried to tell me that my interview scored poorly, and that I would need to retry again in a few months. After some back-and-forth they realized both that it was practice interview and that the person didn't show up. So they rescheduled the practice interview.
When the interview did happen -- this was for ML stuff -- the interviewer was just no great. We spent so much time discussing the differences in terms we used -- mine largely coming from University, there's from I'm assuming their formal education -- that much of the interview was wasted. It literally came down to me deriving what we were not agreeing on for them to understand we were talking about the exact same thing.
I then had to reach out several times for my results, I'm assuming due to the fact that you're allotted one practice interview, and technically I had two(?). When I finally got my results I was again informed that my results were not good enough, and I'd need to wait to reapply. I gently informed them that it was practice interview, and the representative apologized their mistake.
When I reviewed my results... the interviewer didn't rate me too well, largely due to our differences in terminology. They also didn't like my coding style -- even though no one has ever complained to me before -- and rated the coding exercise poorly even though I was able to perform what was asked of me.
I just gave up.
Then several months later, I got an email about being TripleByte certified or whatever.
is that they say they have companies like Apple and TrueCar posting job listings and hiring but they actually never respond and all you get is offers for companies between 10-200 people in size (aka, startups)
I'm guessing this is your experience - was this more than a couple of months ago? We made some changes about ~6-8 weeks ago to how we order jobs to prioritize responsive companies much more than they were in the past. We also use responsiveness as an input to the "Likely To Accept" score shown for each job (here's a screenshot of a posting from my own prod-testing account: https://imgur.com/wSmGCEL).
This was my experience twice trying to use Triplebyte in the past 2 years.
It's a great platform, the indicator letting me know companies aren't going to respond is great.
It just doesn't fix the problem that... the quality of companies on Triplebyte aren't companies most senior/lead/architect/advisory level engineers are willing to settle for. I'm being unnecessarily harsh/biased. I'm sure plenty of people use the platform with great success. I've just come to accept "early/mid-stage startups" job offers from Triplebyte and not much else. Not necessarily "garbage" but... for sure lots of risk.
I did triplebyte most recently in mid 2019 and that was my experience as well. I used triplebyte as well as job searching on my own and I got a few offers from triplebyte, some of which seemed like interesting projects but couldn't offer a level of compensation that would make the risk worth it. I got two high-quality job offers from my own search, both of which were more compelling than any of the triplebyte offers and I am still with the company whose offer I accepted.
The "super week" of onsite interviews was interesting, probably increased my overall interviewing skill, but was probably one of the more stressful individual weeks in my life.
Without passing the minimum bar, how does Triplebyte distinguish itself from other 1000 platforms where people post resumes?
Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Fundamentally, I think this invalidates Triplebyte's business model. Companies don't really care if someone passes your tests. They still put you through LC type interviews onsite. They simply save a phone screen.
I worked at Triplebyte several years ago (but way before any of these changes were being discussed - this blog post is the first I'm hearing of them). I'm an engineer, but I spent a lot of time with the account management team talking to companies. And the biggest thing I remember is how much those companies pushed back on our "rules". They were all eager to talk to the top .1% of candidates, but otherwise just seemed to want an unfiltered firehose of resumes.
So while in theory I like the idea of this meritocratic minimum bar granting special privileges (I liked it enough to join Triplebyte!), in practice we seemed to be fighting an uphill battle with all but a few candidates each month.
On the other hand, I have wasted a lot of time in conversations with recruiters only to be let down at the end by mismatched expectations. I'm interested to see if this new approach can make a meaningful difference in that problem.
> Without passing the minimum bar, how does Triplebyte distinguish itself from other 1000 platforms where people post resumes?
We still have the quiz, and we still show performance on it prominently on your profile (provided you've chosen to share those scores). In fact, we've put a lot of energy into improving the trust companies place in our quizzes over the past year precisely because we think that skills data is important.
> Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Heh, yeah, that's fair, at least to a point.
Our quizzes are predictive. We know this from a lot of objective data, it's what you'd expect subjectively, and companies do tell us that they trust and value that data. That does not mean that our tests have no bias towards certain personality types. Different people respond differently to testing, and that does have a differential effect, not because our tests are bad but because testing is inherently somewhat artificial.
I actually taught test prep before joining Triplebyte years ago, so I've been this first-hand: it was not uncommon for a student who'd been doing well in practice sessions to crumple under the pressure of the real thing. That doesn't mean the test is bad, it just means that some people fare better on tests than others for reasons other than just their raw ability. When we say "likes tests", that's more what we mean: not that that's the only reason someone does well on a test, but that people who perform well in isolated, pressured environments do better relative to their skills than others.
Well, yeah - weren't we pretty upfront about the fact that we were only serving a relatively small number of people? That's not the same thing as "we were only taking good engineers and now we take a new cohort of just bad engineers", though.
I think that goes to another ability. If we need a candidate to gracefully handle a 3 AM production outage, we should expect them to handle an interview.
Who was the woman that wrote about how every "innovative" engineer recruiting platform eventually devolves into LinkedIn Recruiter?
Because she fucking nailed it with Triplebyte.
I used Triplebyte. I really liked it. The leads were not as great or as numerous as I was hoping, and I felt like I still had to do pseudo-technical interviews anyway. Fucking COVID, man.
Fascinating post. I'm personally optimistic on interviewing.io - it's both more useful for candidates (interview practice) and stronger signal for recruiting companies, but this is such a difficult market to break into for reasons Aline describes well.
> We want to stop being a placement agency, and instead become a job search platform that leverages that unique power to create a better hiring process for engineers
> Triplebyte has hundreds of thousands of engineers on our platform, and that means we can flip the script on companies. The collective power of thousands of engineers is enough to change their incentives in a way that individual engineers cannot.
> We can change their incentives directly by rewarding or punishing certain behavior. For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings. When a company’s access to thousands of engineers is on the line, their incentives are very different. The same goes for honesty: a company often has no reason to be honest with any one engineer, but we can disincentivize lying by making their behavior with one engineer affect their access to the next.
So... a union? In digital, online format?
I'm glad that someone there is now realising the advantages of joining forces to negotiate workers' conditions. It was about time high-tech engineers noticed this.
Interesting idea. I love the recognition of current hiring asymmetries. Some of the practices by companies simply shouldn't be tolerated, like the recruiter ghosting, opaqueness on salary ranges, exploding offers, misleading listings or qualifications, lack of feedback, etc., and I wonder if some of these can be solved by a company like TB that facilitates the hiring market and pipelining the process.
So hiring through TripleByte would be conducted in some fixed-length time intervals (e.g. 1 month). Companies specify skills they're hiring for, interested engineers interview for the skills that TripleByte can screen for and supply a minimum salary (invisible to companies), and companies hiring that month bid starting at the min salary for the engineer with all other hiring companies.
If the market determines the hiring timeline, recruiters can't put you on hold if they're interested-but-not-sure, and can't use exploding offer tactics. It also pushes companies to be more honest about salary ranges and required skills. TripleByte would be responsible for providing feedback rather than the companies.
I am a self taught engineer without a CS degree, and I went through Triplebyte at the end of 2017. It was an absolutely incredible experience, and really helped me break into the industry, and I’ll always be grateful to them.
I found an awesome startup to work at. And now, years later, I work at a FAANG company, working on tons of interesting problems. Thank you triplebyte, and good luck with the pivot!
Same here, toward the end of 2015. I would not have a serious high-paying job right now if it wasn't for triplebyte. I'm sad to hear the process hasn't scaled because I think that it'd help everyone out if it had.
> We want to be the job search platform that puts engineers in control.
I love this, but I'm curious about the incentives here. Triplebyte only makes money from companies, not engineers. In the long run, can engineer-centric intentions override a business model based around companies and recruiters staying happy? I hope so!
Companies do pay us. But I think that our real long-term incentives still pull toward building what engineers want, not what companies want. I think that platforms like LinkedIn and Indeed have just gotten this wrong, because they focus on all jobs (not just engineers), and because demand for engineers is stronger than it has ever been. Take a look at this thread from last week:
Our bet is that LinkedIn is going to fragment. They are just not creating a hiring process that most engineers like. People tend to either get ghosted, or overwhelmed with low-relevance inbound (almost no one gets the "right" amount of attention). Companies need to go where the best engineers are, not the other way around. So I think our long-term incentive is to fix these problems.
In any case, I'm committed to giving this a try. There is danger that we get pulled toward building for companies. I want top guard against this by being public about what we're doing, and "showing our work" as we go.
> our real long-term incentives still pull toward building what engineers want, not what companies want.
> There is danger that we get pulled toward building for companies.
A common phrase thrown around with free services is "if you're not paying for it, you are the product".
> For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings.
Maybe I didn't read the article carefully enough, but are you planning to continue charging companies $15k - $30k for the ability to access candidates on your platform?
If so, companies are still your customers. And if you're building and optimizing your product for people who aren't your customers, your real customers (companies) may not be happy with you which will hurt retention, etc.
Maybe I skimmed the article too quickly, but it seems strange to charge companies $20k+ to post a job on your platform, and then actively do things that "force their hand". It might be a net benefit to the engineers on the platform, but I wonder how it will work from a business model perspective since you're potentially creating adversarial relationships with your "real" customers (companies paying you to access your candidates).
Edit: But maybe it's by design, if you actively remove employers who aren't abiding by your philosophy. Although again, that means turning away customers, which means turning away revenue, which in my mind raises questions about the overall business model. A lot of conflicting interests.
I generally agree with the phrase "if you're not paying for a product, you're the product". But the market for engineers is just so lopsided that I think it's less true here. There are a lot of recruiting companies. The only real thing that sets one apart from others is whether they have candidates. So one way to look at it is that yes, we are incentivized to build what companies want, but the main thing they want is for us to have engineers. And the only way we get engineers is by building what engineers want.
That does not fully express my motivations. I am an engineer and find the idea of making the process better for engineers more exciting than making it better for companies. But it explains how I think the incentives work.
This exactly. Their messaging is over the top "we help engineers get leverage over companies!" I have to wonder what Triplebyte is telling the companies who actually pay them all their money.
Triplebyte has always seemed scummy and this obvious--and unacknowledged--contradiction makes me trust them even less.
We do also have differentiating features on the company side. It's not like we're abandoning the idea of building features for companies or anything.
The job search, like other matching problems, is often not zero sum. So building (say) a better search interface for companies benefits engineers as well, because both sides of the market have an interest in a good match. What we're saying here is that in the cases where things -are- zero sum, the power - and therefore the incentives - lie with engineers and not with employers.
I got my job through TripleByte. I consider it my dream job. Honestly, I loved the experience with TripleByte. I got a bunch of interviews to happen at the same time. Having them all together increases our bargaining power. It's hard to get the processes with different companies to coincide if you're just waiting around for recruiters to reach out. I already felt like it gave me the power. I'm sad to hear it's changing, because I would've used it again. I've also referred tons of people out of genuine liking of the service. Not sure if I can recommend it anymore. Still, I'm very thankful for what they did for me. I hope their new style works out.
I'm really glad that the old process worked so well for you. When it worked, it was pretty great. We are losing something with this pivot. The problem is that the old process did not work for far more people than it did. For every wonderful experience with a TM there were 10+ engineers who we screened out. That’s why we made this change.
I think it probably was the point, but I'm guessing the primary issue with it was that it turned out to be harder to profit with that strategy.
To use a rough biological analogy, I think it's generally a lot easier to have a big, rapidly growing company if you take an r-selection rather than a K-selection approach.
Here's an advice. The software engineering market, at least the upper end of it, doesn't need a email forwarding proxy middleman. What it needs is a club-like organisation that acts as a negotiator that leverages insider knowledge to get unreasonable parties on both ends to sign a contract. Good devs don't really search for jobs and don't really talk to random recruiters. They get a steady stream of sales pitches from friends of friends or former co-workers and leverage their fairly wide network to get insider info about companies they're considering to join to get a good contract. The "club" would be like a golf club address book with staff working to connect matching parties. It's surely not a dating site for programmers with ahem.. "AI" selling resumes to data brokers.
I very informally do this right now. I think of my role as a sports agent for top engineers. The engineers I work with pay nothing and the companies who want access to them give us no bullshit, all access interviews where we're not working with HR or recruiters. It works so well for both parties that I'm scaling it at the moment.
Hence the good ol' "We'll give you $5,000 for every engineer you refer to us...and there's no cap!"
At least good companies do this anyway.
Also, executive search recruiters kind-of do what you're describing. Exec leadership is a small-ish world, and the recruiting cycle for, say, a CTO can take months. However, there's a lot of commission on the other end of that. So recruiters basically act as brokers.
Meh. As someone who has moved around a lot this just sounds like friend nepotism. Based on this system I’d be a garbage tier developer since I don’t have friends in high places.
I much prefer the imperfect systems that at least pretend to be fair and objective-ish.
So, I'm personally pretty unhappy with this change, because part of what made triplebyte valuable (and gave y'all the hundreds-of-thousands-engineer userbase) was the _path to competence-signalling_ which avoided credentials that your platform gave, given the quiz -- companies knew that someone being on triplebyte was a strong signal, and engineers had a path to signalling competence that was one-to-many. Eliminating that signal for goals that...seemingly don't require it (why does this new job platform require eliminating the assessment?) seems unwise, IMO
> part of what made triplebyte valuable (and gave y'all the hundreds-of-thousands-engineer userbase) was the _path to competence-signalling_ which avoided credentials that your platform gave, given the quiz -- companies knew that someone being on triplebyte was a strong signal, and engineers had a path to signalling competence that was one-to-many.
On the other hand, when I applied to TripleByte, their feedback to me was "we think your skills are fine, but we want somebody who can perform well in an interview". Which is exactly the opposite of the value you're attributing to them here.
Quoting them, for reference:
> We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured regular expression matcher. It was especially impressive how much you dug into the academics behind regular languages.
> However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview.
This is a perfect example of why we're making these changes (and the problem that came from us being a gatekeeper). There are lots of different ways to show skill. We don't want to be in the position of deciding who "deserves" a job.
In the case of people who needed us to vouch that they deserved a job, yeah. But:
- In borderline cases, we didn't want to risk our credibility and companies didn't want to skip tech screens. So competent-but-not-amazing engineers got shut out.
- Our quiz is not perfect, so engineers who didn't fit what we were quizzing got shut out.
- Not everyone needs us to vouch for them. Companies (reasonably) trust that someone with a degree from a top school and years of prestigious experiences can probably do fizzbuzz, and demanding that they prove it was a barrier to them using Triplebyte.
We think it's better to have more granular ability for companies and engineers to decide what mattes to them. A prestigious company can say "we'll only talk to people who took the quiz and got a top score" (and that score is us saying "this person deserves a job"). One that desperately needs the headcount can talk to the borderline cases or decide they don't want to put the extra barrier of a quiz in the way.
Like most markets, we think the hiring market on our platform works best when it's able to respond to local conditions.
The difference is in the bar for forcing companies to set aside parts of their screening process. To get them to do that - as we needed to under our old model, because we didn't have a notion of "recommended but without the requirement to skip tech screens" - we needed to be making very strong recommendations.
I'm a little bemused at the notion of evaluating someone "strong tech skills, but can't pass an interview" and determining that the appropriate recommendation is "talk to this guy, but -- unlike with most of our candidates -- don't skip the tech screens". The recommendation seems like the opposite of the diagnosis.
How does the new ability to make that recommendation address the original problem?
Because getting some leads with less certainty is better than getting no leads with higher certainty in a tight market? Wasn't this addressed in the post?
I'd say that main value was that we opened doors for people (got them opportunities they would not have been considered for without us). I don't think what we need to be gatekeepers to do this. Yes, not everyone can succeed (get a job at at top company). But we can help everyone show their skills in the way that's best for them. We can fight ghosting and lying and create a less hostile process. We're keeping our quiz (so that people who do well on tests can get opportunities that way), and also creating a job search process for people who do not want to do a quiz (the majority of engineers). The idea is that they will show their skills other ways (past experience, side projects, open source work). By us not being a gatekeeper we open these other paths.
I feel the exact same way. I went through the interview process with a handful of good startups through Triplebyte (and got offers from almost all of them) and even though I ended up taking a job with a company outside of TB, I still walked away extremely impressed and confident that I'd use the platform again in my next search. I loved that I was pre-qualified and didn't have to intensely prove my technical chops at every on site. With this change, I don't see the benefits for me to use TB over a competing job board or just applying to companies directly. With these changes, I don't plan on using TB for my next job search.
Edit: just saw the comment that TB is keeping the quiz. Would consider using the platform in the future as long as that pre-qualification track is available and that this new direction is just a superset on the existing TB features.
this seems like it's just diluting the signaling value of being on the platform? (and seems like it'll quickly create an upper and underclass of quiztakers vs nonquiztakers)
There's some risk of that. But I think that we can make it work. Basically, the model I have is that there are many different ways that an engineer can signal competency to a company and get a job. One way is doing well on our quizzes. But another way is having impressive experience at a good company, or a lot of open source contributions. Any one engineer might be really good, but look bad on one of these metrics (someone who gets stressed and does poorly on tests, or someone who has mostly worked for the government and does not have side projects). This happens a lot. When we ONLY ranked engineers by our assessments, we just missed people. The goal now is to let people show skill in any of these ways.
Well, I hope you're right and will keep an eye out (and will regardless be Fasttrack-ing in 6 months when I'm looking for FTE work, so I'll see firsthand :p)
A recruiting agency with a hundred thousand recruits in a database is not functionally any different than a call center with a list of leads they purchased and validated.
Do not believe anyone who tells you they can force employers to change their methodology when it comes to salary and compensation revelation. Recruiting agencies are so far outside that wheelhouse that it's damned near scandalous to even pretend the statement has any validity.
Also, I think it's important to differentiate staff recruiters from outside recruiters.
Staff recruiters are HR people. They're on the job to shove square pegs into square holes.
Outside recruiters are salespeople. They're on the job to explain why a round peg can reasonably fit in a square hole.
This is your career. So act like it.
If 10 different people were each going to give you a hundred dollars if you described yourself to them in a way that would obviously ingratiate yourself to them (person with cat photo, person with dog photo, person with photo of their kids, etc) most would immediately come up with reasons to make themselves1 appeal to their tastes.
But people keep on submitting the same dumb resume over and over again and expect a different result. Accentuate the things in your resume that matter to the company you're trying to get a gig at. Yes, for each of them. If you would do it for a grand, you can do it for a career.
Interesting idea. But it seems like there’s room for abuse - won’t engineers, who are unhappy that they didn’t get an offer, just report companies? Don’t you just become Yelp in that case, full of reviews from salty customers? How are you going to get useful data when you’re worried about that?
This is certainly an issue. As you say, it’s a problem on yelp and glassdoor. It’s why most nightclubs have low yelp ratings (people they exclude give them low scores). There are a few things I think we think we can do:
1) Look at relative data, not raw ratings. Written reviews or raw numerical scores are what show the nightclub effect the most. What we care about is a ranking of companies (showing the better companies first in search results). This may not be as impacted by the problem. If sour grapes from people who failed interviews overwhelm other signals, we can only look at the scores from people who pass (or normalize the two groups separately).
2) Ask very specific questions (and maybe provide dummy options to attract sour grapes). I suspect that questions like "did this company say they could meet at $250 salary and then offer you less?" will get more accurate answers than "does this company have a toxic culture?". (We do actually want to get culture data, but I want to be really specific about the culture questions we are asking).
3) Use objective facts, not subject opinions. For example, we can tell when a company ghosts a candidate (because they reply through a proxy email that we control). So we're listing the 'ghost rate' for each company.
> I suspect that questions like "did this company say they could meet at $250 salary and then offer you less?" will get more accurate answers than "does this company have a toxic culture?".
I'm pretty sure many salty people will just lie anyway, to punish the company. Source: I've been on the internet a long time.
I think this model of 'punishing' companies would've worked when you had a high bar for being on the platform in the first place, but if anyone can get on there, you'll have a lot of low quality candidates who are very mad that nobody will take them, and they'll overwhelm the reviews.
Remember that there's a survivorship bias at work: the lower quality people will take longer to get hired, which means they do a lot more interviews, which means a lot more reviews on your platform. Essentially, it's a reverse meritocracy when it comes to evaluating companies.
IIRC I wrote about 30 custom cover letters to Triplebyte companies... about 2-3 responded. My response rate to cold emailing/applying to ~10 F# companies is literally 100%. To be fair though, the TB applications occurred right at the start of Covid, and my F# job applications well into Covid.
Anyway, a ghost rate would be greatly appreciated.
In my experience toxic companies will essentially force employees to give good feedback through either social pressure, incentives or threat of repercussions. You can see this quite clearly on Glassdoor where a bad review is followed by five very similar good reviews within a week for some companies.
In 2021, you don’t even have to search for engineering jobs. You either start out with a shortlist of companies and roles or a recruiter reaches out to you and kicks off the process.
Maybe this would be useful for folks just starting their career, but on the hiring side of this, in the last year I haven’t made an offer to a candidate who didn’t already have at least one other offer on the table.
Overall I think there are a lot of misconceptions about how people are hired from the recruiter side — which is why most of these platforms see the recruiter as the target customer, not engineers. I get that y’all are trying to change that, but without insight into the hiring practices at any given company (e.g. my company gives a lot of latitude for salary negotiations but every hiring manager negotiated differently) it’s going to be hard to pull a signal from the noise.
In a narrow sense, that's the thrust of the case we're trying to make here: if veteran engineers didn't frequently have competing offers, they wouldn't hold the amount of power they do.
That being said, the experienced engineers who can get that kind of attention have other desiderata in their job search beyond just "get an offer". "Get -an- offer" is a goal more typical of a new graduate or someone trying to break into the industry than it is a goal of experienced engineers.
To people who are sure they can get a job, there's a big difference between a 150k offer at a company that has none of the workplace traits they want and a 200k offer at a company that they'd genuinely love to work for. And right now, they can't necessarily tell the difference between those jobs before going through a bunch of application steps. The example we mention in the post - of a highly credentialed and very senior engineer who kept having his time wasted by recruiters who refused to admit that they couldn't offer what he was worth - wasn't cherry-picked. It came up organically while we were doing user research, because it's a really common problem, and was echoed further by a number of other senior engineers we spoke to.
(Of course, we care about the folks just starting their career, too, so I don't want to totally ignore the importance of solving problems for them. But I think we are solving an important problem for senior devs here, too!)
While the constant supply of recruiter contacts and even direct internal company contacts I get is nice, I repeatedly see the sort of basic issues as mentioned in the piece of concealment of or lying about pay (or incredibly broad pay ranges), entirely leaving out information about a job's location even when it's on the other side of the country, trying to offer entirely inappropriate junior positions that I'm never going to be interested in... all stuff that has zero cost to a hiring manager but wastes my time. I could see a lot of usefulness in something to push back against that in a structured way.
On the recruiting side (I’m not a recruiter but a director making a lot of hires), we actually do the opposite all the time — we post relatively senior positions then downlevel whoever we hire to fit their experience level. A lot of people think more highly of themselves than the employment market does.
We look at the job posting as a brand marketing — all we’re really trying to do is get you to apply and fill the funnel. The whole hiring process is designed to be a calibration on role, cultural fit, compensation, etc.
We have the conversation about level before we give an offer. It’s genuinely because we don’t want to put someone in a role they’re going to struggle with and be unhappy or burn out. I would much rather underlevel someone starting out and promote them 6 months later than hire them at a higher level and then needing to have the conversation about performance early in their tenure or cut their bonus.
I work in consulting so high turnover is just expected. We’re actually a lot lower than the industry average. The job isn’t for everyone, but there are people who thrive in this environment. For everyone else who isn’t a good fit, we get a year or so worth of billable hours, they get exposed to a lot of different scenarios, and we give folks a lot of runway to find something they do like.
It’s just the nature of the beast though. If I get a candidate who is legitimately at level, I bring them in at level. There’s no need to be shady about it, but job search sites are what they are and that’s the game you play.
Does anyone think that this will work? Call me a cynic but I have very little faith in reputation building companies /platforms which know which side one's bread is buttered.
Anecdote: Recruiter R from Company A reaches out to me, we have a conversation and she says HM (hiring manager) loves your profile and wants to talk couple of days later and so we schedule a time slot to talk to HM. Next day, R calls and says HM wants to know why I left a previous employer X and I just blurted out the truth which was that X made it very difficult for me to take FMLA when my kid was born and I got pissed and left. Shortly after this I received a rejection message. This is classic discrimination as defined by EEOC.
I replied to R and said this is discrimination and she denied it. I posted my experience on Glassdoor and my post was flagged and removed. Nowadays I am honest but not truthful.
After being enticed to take their quiz and achieving perfect or near perfect (I forget now) generalist score, I was marked as hotlisted for a tech interview position with them. Keep in mind, they solicited me for this.
No response after getting the "wow, nice quiz result" email. No further emails. No replies for months to my gentle inquiries. Eventually they did reply, saying they were sorry but they didn't need help.
I have significant doubts that they were hiring tech interviewers in the first place. I think instead they were gathering market data at our expense.
Furthermore, there were detailed stories from users on Glassdoor about their internal policies and attitudes, mostly painting their leader as lacking scruples and leadership ability. Perhaps this was all fake, but if so it was really convincing and thorough.
I'm curious if anyone has had legitimately good experiences with them.
Triplebyte was a dud for me. Way too many garbage startups recruiting.
I have started to re-write my resume for each application and that has yielded much better results in terms of quality of the company and number of interviews.
If you are submitting the same resume to multiple companies, you are doing it wrong.
FWIW, this isn't really true as you get some experience under your belt and have a stronger resume (meaning, having worked at companies that people know of with a clear story of how your own role made a big impact there).
But glad you figured out an approach that worked for you!
Will this newfound respect for users' privacy be backed up by a contract or some other legally-enforceable commitment, or do we all have to worry about when you decide to pivot again in another couple years?
I hadn’t thought about this from a legal contract perspective. I like the idea of making our commitment enforceable. Do you have ideas of what you'd want in a contract?
The candidate me likes cutting out some of the bullshit involved.
The hiring manager side of me on the other hand continues to remember that mishires are expensive. I've also very skeptical in that once a month some staffing agency comes along, tells us they have some fancy heuristic for matching good candidates to our jobs, and then unleashes a tsunami of mediocre leads into our recruiting database. They'll then proceed to act upset and surprised when we don't want to intervene sixty percent of them and most of those we do interview wash out. I guess TB can somehow promise that a certain baseline technical competence is present, but even that's of moderate value. If I really need a candidate who is skilled at X, I might be willing to sacrifice some skill in Y because I already have two engineers strong in Y. I never found TB sufficiently granular in how you configure the automatic screens to allow for that. Secondly, it's totally useless in soft skill assessment.
The cynic in me just sees a lot of this TB now trying to force you to expend internal resources to rank the unqualified people or people whose skill sets don't align with what you want (forcing you to respond with data on how you handled the application) because they can't crack the nut of doing it themselves. So they're going to leverage the customer's recruiting teams to code the data for them.
I'm salty towards the platform because they rejected me a few years ago, and my ego has barred me from using them again despite their outreach. I was also very unhappy when I heard about the public profiles. I don't want people to see that I interviewed poorly, or to have any kind of public record of that. I'm a decent engineer, I swear!
I had some bad luck. My nodejs build broke after a recent update on my machine, which I didn't realize until right before the interview. During a test with a different language, I tried to define a constant with the same name as a built-in function and ran into a vague compiler error (something about missing parentheses, ugh). This language has case-insensitive function names to compound the confusion. I unfortunately looked up how to define a constant in the docs. Their conclusion was that "I was uncomfortable in the language," despite having used it for 10 years. There was some other feedback which I felt was inaccurate, I think I just had a bad day, and obviously didn't convey my knowledge and experience well. I could see why a recruiter would hard pass on me for some of the stumbles, since their main goal is to forward candidates who interview well. It hurt to get rejected.
I am not sure who are these engineers that Ammon talked to... but nowadays what most engineers I know do is visiting Glassdoor or Blind. There you can anonymously and openly discuss salaries, their opinions about companies, business units, teams, and even discuss interviews.
If a company has a bad interview experience, a bad culture, or the management is impopular, if the work life balance is bad, or anything... you'll be able to read about that in those websites.
For example, a recruiter contacted me last year regarding an opportunity for a fancy senior role at a large company (Sr Staff engineer)... I checked the Glassdoor reviews for that company and found many testimonies from people who were contacted with the exact same narrative, but at the end of the interview process they were offered vastly lower levels, so clearly that was their intention all along. Because of that, I assessed that interviewing there was going to be a waste of my time and I ghosted the recruiter. Developers win.
Of course, you need to take some reviews with a grain of salt, but if everyone is saying the same thing, you will at least have a sample of what you are getting into.
Having said this, I am not sure Triplebyte will ever be able to effectively solve or mitigate the problems Ammon is describing. It is possible that he needs to continue talking to developers, and I am not trying to say this in a rude way. There are also many more problems that need help fixing that Ammon did not describe.
Having a conversation with a developer about different topics can give you some insights on the breadth and depth of the knowledge of a developer, and many interviews omit this. I think interviews should be "adaptive" tests, that get as progressively more difficult until you reach your limit.
The difference is that I think we're in a position to actually change behaviour (require companies to not do bad things like ghost candidates, lie about salary).
The original TripleByte worked well for me. Even though I didn't have a stellar resume, I got the chance to talk to some really interesting companies. My TripleByte recruiter was helpful in deciding which interviews to take, and there was mutual fit at all the companies I went on-site with.
I recommended TripleByte to several friends whom I considered reasonably skilled and currently undervalued. None of them passed the test, however, so they got little value out of service (except for the interview feedback, which at the time was unusually excellent). And I ultimately needed TripleByte less than they did, since someone who could pass the old TripleByte test also has a high chance of eventually passing on-sites at Google, Facebook, etc.
So I understand the pivot, although it will certainly be harder to differentiate yourselves in this new space. Good luck.
A lot of research on interviews has shown they basically don't work. You just can't grill someone for a few hours and know how they're going to do on the job.
For unemployed candidates, we should do away with interviews altogether and do paid 2-week trials instead. After the first 2-3 PRs it's pretty clear if someone has the skills you need and to what degree. It would also help the candidate decide if they like the company.
Employed candidates are harder to evaluate and I don't think there's a single formula that will work for every company.
Triplebyte as an idea is great. As someone else pointed out, they are obviously not doing well financially, or they wouldn't have pivoted. I think their pivot will not work, though. They have gone from a unique thing that saves engineers lots of time (in theory at least), to another job board. How many of those do we need? How many companies would even want to partner with a place that forces them to give up so much of their own power? I don't see this one working out.
I applied through Triplebyte in 2015 & again in 2018, and got to know Harj & Ammon a bit. They, along with Gilluame, all struck me as remarkably smart and incredibly dedicated to their vision. So if after 6 years that lot haven't "cracked the code," that's reason enough to abandon hope for there ever being a sane recruiting process, anywhere.
My takeaway from my experience finding a SW engr job was that actual technical capability was really only a second- or third-tier consideration in companies' hiring decisions. (This is often misaligned with the companies' actual business needs, but that's not what materially drives hiring decisions either.) So, although TB nailed TFOO the competence assessment & pre-screening problem, Amdahl's Law still applied to the whole enchilada.
What TB should do is monetize their experience placing candidates over the last 5+ years and just sell training & maybe some standardized SaaS products to HR departments -- especially at startups! -- that regularly need to hire SWEs/SDEs (yes I'm a FAANG-er now). Because the real dysfunction in tech hiring originates & thrives in those departments, and...something about robbing banks because that's where the money is...
I almost applied to Triplebyte, but at the time they were Bay Area only, and I needed a remote job. The most important part of a job search is to avoid wasting time, so I stayed away. (But I would have used them if I was in the Bay Area.)
What am I looking for?
1: I want my skills evaluated in a standard way by some neutral licensing board, just like engineers who build dams or surgeons.
2: I want job interviews based on mutual interest, not proving competency.
3: I want my colleagues to be certified, just like me.
4: I want legal standards in place that give me leverage against employers who are cutting corners, or managers who "know enough to be dangerous."
Example: My wife, a doctor, had to go through an intense process to be certified. The vast majority of medical school graduates pass certification. She only applied for a handful of jobs. The interview was based on mutual interest because the certification process proved she was competent. All her colleagues are competent, and her hospital is well run. There are legal guidelines that make it hard for her hospital to cut corners or make her do something unethical.
If your wife screws up anything significant, someone dies.
For most developers, the worst they can do is bankrupt the company they work for, and even that usually requires multiple compounding screwups and a bad luck roll.
I personally think the gap between our standards for doctoring and our standards for software development makes a lot of sense.
I do think people building life-sustaining software should be held to a higher standard, but that's pretty rare.
Wait, so... Where's the money going to come from? Unemployed engineers wanting to find work?
Also, are we sure there is a shortage of engineers, or simply a shortage of engineers that can pass the hiring interviews and qualifications?
I think a lot of companies don't always mind waiting to find someone who passes there test, be it a newer grad, or an existing engineers that has now prepped up better.
It's sad but interesting to see the pivot that Triplebyte is taking. As the top comment says, if they were doing great they wouldn't be pivoting, so it speaks volumes.
I still don't understand exactly what benefits it brings to me as a programmer though. I'm about to start preparing for interviews with 25+ years under my belt, after not working for 2 years and I've accepted the double-edges nature of these interviews. I know I need to LeetCode for 3 months to be competitive and memorize as many of the answers as I can. I know that some companies will ghost but others will be forthcoming. I know that in big companies, different teams have different recruiting processes, so it's hard to put an entire company in a single basket.
I have friends at all FAANG, and they are very open with their salaries and TC, so I don't know what Triplebyte would bring to the table besides friction, so be perfectly honest.
I like this new direction but the stated power dynamics don't align with their history. The most popular kids in the room (FAANG) define the hiring process and some of them outright refuse to skip steps, which hurt Triplebyte's reach. If those co's get hidden on TB don't you think a ton of engineers will ignore the platform?
In addition, as my username suggests, I really like the idea of a representative for Engineers during the job search process, but I also want to make sure I'm the customer of this representative. If I'm paying TB then I think the incentives are aligned. If they're just a job board then their new stance is opposed to the interests of their
true customers (hiring companies) and that won't work out too well for them.
I saw this change comming in the market. Every company which went IPO want to double the strength of the employees. I have a friend who got response from companies like Compaas/Affirm/Coinbase/Sofi also (Apex clearing after NSTB merger want to double it) and a lot more.
My firend cracked interview and joined one of this and for others like me have no traction (6 months in) on the resume at all after using hired/linkedin/direct careers/cold emails. Doesn't matter you have github with real projects/portfolio with good responsive design .......
If this process is broken and we also don't know what are we doing wrong then how all them are going to find all these talent they need??
I tried the triplebyte test when I was finding a job ~2 years ago. The test was biased towards web developers (which I was not), and I failed it. It was an infuriating experience and turned me off to the whole concept. I went on with my normal job searching process (which generally involved recruiters who reached out to me, and friends giving referrals). I landed a very nice job at a big tech company and swore to never touch triplebyte again. This is just an anecdote, but I would bet that many highly qualified engineers failed the test once and then they decided not to ever go back. If the test had told you why you failed or what to study, I bet I would have tried it again.
Right, so the companies are still your client. But you somehow think you can strong-arm them because you have some CVs in your database. In the end, the job platforms end up serving the client's needs. And company stay the client.
The concept sounds interesting but the execution seems lacking. I couldn't complete my profile because of spurious "check your inputs for errors" messages when I tried to click save on the very first section. Reloading the page several times didn't help though eventually a hard reload did. The project section silently lost an entry when I clicked save, presumably because I entered a year without a month. In the Java quiz, a question about generics failed to properly escape the angle brackets, making three of the four choices indistinguishable without reading the DOM. Question quality was quite variable.
I did a triplebyte interview once. I come from a Computer Engineering background, which was more or less spun off from Electrical Engineering. We learned a lot about low level stuff and signal processing, not so much about discrete math and algorithms. For me their interview was focused on Computer Science and Web development, which I didn't really do.
The interview itself was nice and ok, the feedback was really appreciated (after all this more or less reflects what high paying programming recruiters are looking for). But I felt like there was no place for people with my background.
Lots of astroturfing going on here. HN community is their targeted audience, so they're hammering this place hard with positive spin. This kind of aggressive marketing makes me really suspicious.
Triplebyte was worthless from the start. I, a non-developer, took their quiz a couple of years back. The exact wording escapes me though the gist was "You've done exceptionally well and have 'x' chance with interviews." They didn't give any details on where you actually stood. The quiz was taken as a joke as I know java and objects exist, but no clue how to use them so it appears everyone gets the same canned response which would result in some people receiving false hope about getting a great job.
> Over time, these problems compounded. Engineers who we rejected held that against us (which was completely understandable).
I was rejected when I did the quiz at university, trying to get a job.
What made me a bit angry is how happily they rejected me with an automated email, after I put effort and time into doing their quiz... I didn't feel like recommending them, or even visiting the site ever again.
Glad to see they're finally aware of how this approach doesn't benefit them as much as it could of if people were still invited into the system.
One of the problem with recruiting is not just power imbalance but information asymmetry (or even plain lack of information).
What Triplebyte is trying to do is give more information back to job seekers. But it will be limited to the companies they actually work with (and punishing your own clients sounds like a peculiar business model...)
Why not build a Glassdoor for hiring, that would cover the entire industry? Anyone can go and describe the hiring process they went through, if they heard back or not, etc.
Triplebyte was an interesting experience for me the time I tried it out. I got lots of calls and several onsites, but my lack of professional experience bit me on every single one of them.
I felt like Triplebyte worked hard to create a good experience for me, but that overall the companies I interviewed with just viewed TB as another placement company. Other than getting to skip the fairly easy phone screen phase, I didn't feel like I got much value from the experience at all, and I had to put a lot of work into it.
I would probably take this down if it was me but I can also appreciate TB's radical transparency and putting themselves out there for public criticism.
Ever thought of building a recruiting recommendation software product, instead of just selling people? You have the data.
edit: I want to be more constructive with my feedback. You have four years of data on hundreds of thousands of candidates. I don't know what the legal implications are regarding using that stuff as training data, but.. embeddings. doc2vec. Thank me later.
The only good thing about TripleByte was the structured quiz and interview process. Everything else--inability to edit key portions of profile, the most boring set of tech jobs imaginable, very few companies on there--was a joke.
The quizzes weren't quite a fit for ML, nor was the interviewer or job density there for the domain, but the immediate half-hour call with the promise I'd passed a technical screen was extremely efficient, even if Triplebyte lacked any basic level of market liquidity.
I've always thought triplebyte was interesting, I did a first online interview with them while finishing school but basically had an offer from a large company in hand by that time and haven't looked for a new job since. I think they should try and pull people who indicate they are interested into attractive job searches, I mostly care about compensation as long as a company is decent.
As someone who has been critical of TripleByte here on HN in the past, I appreciate the identification and acknowledgment of where things went wrong and why people were unhappy. This seems like a sincere pivot where the community was heard. Wishing TripleByte luck and success.
I can also report I haven’t received a single TripleByte email since the issue was last brought up :) Thank you for standing by that.
I had a great experience with Triplebyte's quiz and interview. I suppose I'm in the category of people from "nontraditional backgrounds"; I majored in English in college. Getting past resume screeners was an enormous relief and an important boost to my career. That boost will have compounding effects for the rest of my life; I love the job I got through Triplebyte.
> Triplebyte has hundreds of thousands of engineers on our platform, and that means we can flip the script on companies. The collective power of thousands of engineers is enough to change their incentives in a way that individual engineers cannot.
Copying this from the article. Doesn't this sound like a union? Unions generally lead to mediocrity though.
I haven’t actually gone through the TripleByte process, but my impression of the value of the platform to engineers was that only the at least minimally capable ones could be on there.
Does this not mean that even if you apply from TripleByte that you now still need to go through FizzBuzz and all that?
> Going forward, our assessments are a purely optional means by which engineers can show their skills to employers (who overwhelmingly tell us that they trust our scoring)
So it sounds like the hiring manager can choose to skip a fizzbuzz call when they know you already passed.
I like this direction and I'm rooting for you now but it's tough when I can't even fill out the Basic Information section because of a bug in which the form won't submit even though there are no errors.
Another voice in the fray, I also did well enough to bypass the technical screen. Interviewed with four companies they recommended. Picked one. Still with them two(-ish) years later and an outspoken advocate.
It's a tricky situation because you have to balance being accepting of people with non-traditional backgrounds while also making sure they can do the basics. I liked that TripleByte could condense DAYS of interviews into a few hours.
In general, long tech screens are hard on people that can't afford to take off a full day, so it's tricky for people already working in crushing jobs that need a change. The TripleByte quiz worked nicely for this.
On the flip side, if their tech screen is filtering people for reasons other than ability, it's removing possibly good candidates who deserve a shot at something.
The new approach is sounding like it's trying to satisfy the latter, which is good for improving inclusivity, but won't be to everyone's liking.
I found Aline Lerner's assessment of new age recruiting services including Triplebyte to be informative. She may be wrong, and you may not agree with her take, but I do appreciate that she's not trying to bullshit the reader.
"Triplebyte, to my mind, did an admirable job of trying to solve the credentialing problem. But their approach was not without shortcomings: 1) not everyone wanted to take their lengthy quiz, even though the quiz was well done, and 2) scaling up an army of interviewers, all of whom had to be trained in exactly the same way, was non-trivial and not cheap. These challenges were surmountable, but the challenge that wasn’t arose from the second issue that all companies had to face: lack of candidate autonomy, which was driven in part by a lack of faith in the credential.
Once you passed Triplebyte’s assessment process, just like at Hired, you had to interact with a talent advocate (Triplebyte labeled them “talent managers,” but again they’re just recruiters). The talent manager would examine your background and short-list you for some companies of their choosing, where you’d then go onsite. You could have some input into which companies you spoke to, but it was limited, and if you didn’t meet the company’s (often somewhat arbitrary) criteria, no matter how well you did on the assessment, its doors were closed to you.
As with Hired, having an army of recruiters AND interviewers working for you makes achieving SaaS margins impossible, and then, you’ve essentially become a tech-enabled recruiting firm (albeit this time one with much better performance data!).
Just like Hired, Triplebyte eventually moved away from the auction marketplace model, fired most of their talent managers and interviewers, and fulfilled their destiny, becoming a glorified LinkedIn Recruiter clone. Recruiters could search for candidates, just like on LinkedIn Recruiter, based on their pedigree, languages known, and so forth. One thing Triplebyte still does differently, however, is to leverage their aforementioned coding quiz to annotate candidate profiles. (Presumably, they are using their historical interview data, from when candidates had to do BOTH, to predict how people will perform in interviews.) The limitation, of course, is that great people will be unlikely to take the quiz in the first place, especially now that it no longer fast-tracks them to an onsite."
Should I bet that this pivot will also fail. I will suggest you go on a road trip outside SV, talk to company HRs, small company and startup leaders, education providers, engineering student, and entry to mid level engineers. Also broaden your “engineer” scope beyond software.
“ In short, there is more demand for engineers than there is supply, and that makes engineers powerful.”
This statement is very telling that TB leaders got tunnel vision, a very SV, startup, software centric tunnel vision. Let me rephrase your statement to what you really meant.
In short, there is more demand for “experienced Software” engineers “in startups “ than there is supply “in Silicon Valley”, and that makes engineers “,with pedigree of FAANG or top schools or well connected,” “very” powerful.
So, from what I understood from the article, the value of Triplebyte to the employers is that they are pre-screening the candidates. How are they planning to keep this value if they stop doing it?
Credit to these guys for not giving up. Hiring engineers remains unsolved so if they're still in the game they still have a chance to ultimately solve it and reap the rewards.
The best thing I got out of Triplebyte was a great piece of written negotiation advice, from a recruiting specialist that has, presumably, since been laid off.
This simplifies to incentives. Who pays Triplebyte? STILL THE COMPANIES.
So unless there is a simple business model shift to charging the candidates, all of their words are empty. THIS is why they made the mistake of making profiles public, because they were thinking about their customer (THE COMPANIES) and not their product (THE CANDIDATES)
"If you're not paying for it, you're not the customer; you're the product being sold." Tim Oreilly Tweet
The biggest problem with engineering job searches is that they're a huge time sink. I'm a senior engineer with Google on my resume, so I don't have much trouble getting in the door or passing interviews, but the process is such a slog. If I were to take PTO for a week and cram my schedule full of job search, I could probably get 3-4 interview processes done per week. That's not a lot, considering that it's good to line up offers and that many people have limited PTO. Even if your PTO is "unlimited", it looks suspicious to be taking random days off for on-sites.
What this means is that I try to keep doing my job and schedule phone screens and recruiter calls in the gaps when I know I won't get meetings. I try not to have to block time on my work calendar for these in case someone wonders "why does Troy suddenly have so many random meetings all of the sudden". The scheduling process with companies is a pain, they ask for 2-3 time windows for the phone screen, then often take a couple days to pick one. So I have huge time blocks reserved for them that I can't give to other companies I am interviewing with. It's stressful to try to slot all this in. Add to that the fact that recruiters want to do everything over the phone (which is very unnatural for me), so I have to also schedule little interruptions for 15 minute "chats" to "check in" and prepare me for the next round. I appreciate these things but at the same time, I'd appreciate fewer of them.
When it comes to the actual interviews, some companies seem to be fine with one phone screen, some need 2-3. Sometimes they are an hour, sometimes 45 minutes. Some companies see that I work at Google and deduce that I might be competent enough to skip the phone screens entirely. I appreciate that.
It's the same with on-sites. Some are a half day, and some are more of a full day. It's not a big difference in practice because I couldn't really take a half day off from 11-3. Many companies have a "no interview day" which is great for their employees but hard to work around, particularly if it falls on "non-suspicious" PTO days like Monday or Friday. Sometimes I wish we could just do several phone screens instead to make scheduling easier.
I'm not sure how to solve this, but it's the biggest pain point by far and if someone could solve it I would absolutely use their platform. I want fewer phone calls, fewer hours of interviewing, and a clear understanding of the time commitment ahead of time.
I also wish companies would be up front about WLB so I don't waste time interviewing, only to learn they work "maybe 50-55 hours a week". That's not a "little more" than 40 guys, that's more than 6 full work days every week. You can't just ask at the outset because some people perceive that as a red flag.
The TLDR says : "There is far more demand for engineers than there is supply, and that makes engineers powerful in a way other professions are not."
I can't believe this. I must be a garbage engineer then, because I found the US job market for software engineering roles to be absolutely soul crushingly difficult. I am not the only one, my friends did too.
I have given up several times and then restarted job search. I have been grinding leetcode but companies want nothing less than complete perfection.
I actually thought something like triplebyte would be useful, give a test once and have the score be valid for a couple of years, like a GRE for algos and data structures.
I'm not sure a middle man is needed like this. Can't a company put a job ad on their site, publish the actual salary range, and make the interview process as short as possible? Standard job boards can help aggregate this.
The "solution" to the big problem might be a 10 page PDF (for reasonable fee or free) to engineers on how to play the game to their own advantage. Of course no one really profits big from this enough to keep a startups lights on. But if all developers played the game well it helps everyone, even the few that don't play it (like a vaccine).
I clicked into this article knowing nothing about Triplebyte (not meant to insult), at all, and after reading the first paragraph, I'll admit I jumped to a whole bunch of conclusions, the worst of which was probably:
I wonder what 'fad-diet' solution this company I've never heard of is pitching for either (A) getting me an engineer or (B) getting me a job as an engineer. Oh, let me guess, you're going to find those magical engineers who come from non-traditional backgrounds or magically making it possible to both "identify the top 5% of Ruby on Rails developers with no fewer than 2 years of React Devops Cloud Experience"[0].
Going from that, if you RTA -- I was pretty horrified to hear what they thought would be effective in this space. I realize you'll have to take my word for it that there were literal sentences in there spoken as "good ideas" that would have literally had me thinking "oh, so this company's some form of scam" had I happened upon them before they chose to reverse these ideas. I gotta say, public profiles, entrance exams in an attempt to "hack hiring"[2] don't know what this company is. I have to take everyone's word for it that they're somehow big in this space as I haven't looked for a job, nor been responsible for finding viable job candidates[1], so that's probably me.
I was kind-of torn about his apology. It was really un?-professional (I mean that as a compliment but I can't find a way to say 'not corporate-fake' either in 'sorry-not-sorry', 'deflection' or with what appeared to be much concern for what the lawyers have to say on the subject. I don't know that the lawyers would have to say much on the subject but I'm told that this is often the reason why companies manage always manage to communicate "Sorry,... (assholes...)" every time they apologize. There's always that miserable thing, they're going along "We apologize to our customers ..." it's looking good "... for the inconvenience this outage has caused." Seems simple enough, except you've taken no responsibility -- the outage was to your software, or your hardware, running on wires that you pay for which you are supposed to have redundancy for, and all of this stuff is outside of your core competency but there are many companies who can help if you're willing to spend. And I'm sure when the issue happened, there was a key choice that had you spending a large amount of money and me getting a flight home; which you also chose to inconvenience me over.
But Triplebyte wasn't Southwest. I think the author went a little over the top on the apology, almost crossing into the "trying too hard", except that the things he was apologizing for -- basically, being blind to some pretty obvious problems -- were kind of spot on. Please don't think I'm trying to throw some sort of slant compliment; I realize I sort-of sounded like a dick.
There's no malice in that statement; I've been more blind to things I should have know that were right in front of me than a silly business model decision made too quickly. I am certain I have not apologized this clearly/bluntly.
And the thing of it is, if it hadn't been done so well, I probably would have read the first two paragraphs, closed the tab and made a mental note to never touch them -- basically, having them go from "I don't know who they are" to "probably a scam". It's the risk a company takes when their apology gets attention beyond "people who (currently) care". I am not currently hunting, but I'm all around it on the "need to hire someone" side and I have no guarantee I'll keep my current job, forever.
Thinking about it -- I've now read a very long post and written a very long comment about a company that I had never heard of and upon hearing about them, couldn't care less about their services (the space, yes, deeply). They'll probably be the first address I put into the bar if my needs change in the next year or so because I can't name a single other place I'd actually look. This guy might be an evil genius; if I suddenly have a strong desire to change employment in the next three months, I call witchcraft and there'll be pitch forks and hay, or...something. :)
[0] I must have been in a really bad mood because HN has never been a place that promotes that crap to the level of this post, so that should have been my signal that there was something of value here, and I'm glad I stuck it out.
[1] Mostly only interviewing; we have a pretty spectacular individual who has extensive knowledge about hiring for the rather broadly scoped kinds of work that just my tiny team in the 100-or-so company that employs me. I have a process that, though I am always tweaking, the major points are about as perfect a process. I've commented like this on that subject many times in the past if you want to be super bored. :)
[2] I'm confident in my abilities as a software engineer; push comes to shove, there's many, many ways to make enough money to live reasonably in most parts of the US. I've never had problems making money writing software, either directly for an employer or on contract, independently. Neither my wife nor myself have completed a 4-year degree (though I was one semester away; long story), she did a job that does not require a degree -- became a Realtor -- before we met, six-figures in 2003? I think. The most Senior guys on infrastructure pulled that where I worked (they paid a little under average for the role/area but like 1-2%).
tldr: they're pivoting from placement agency to job board
i'm not sure about the viability of their "search criteria" though - things like test coverage, release cadence etc are team specific, so it's hard to define it at a company-level, other than maybe small startups
my advice for triplebyte is to partner with leetcode and establish an industry-wide coding certificate, kind of like those SAT or GRE tests that you pass once and apply everywhere.
"Jesus this guy writes and writes and says nothing. Wait, there was a link at the top to some sort of TL; DR, let me check on that to find out what the announcement is"
> What are we pivoting towards? To explain, let’s talk about economics for a second.
What's bad about that? It's obviously testing understanding of scoping and side effects. It's not hard if you understand those things and confusing if you don't, which seems like the kind of question you'd want on a "general coding assessment", no?
The only thing that bugs me about that question is that "the output of the following function" is confusingly worded. It's the sort of question that a competent candidate might get nervous about and start to overthink: wait, that isn't a function, it's a code snippet. The function here is foo – so maybe they mean "what's the output of foo"? But then what does "the output of a function" mean? I suppose they mean return value? Is this a trick question to see if I remember what push returns? Damn it, does it return the entire array or just the pushed element?
That's what my mind usually does with questions like that and I'm far from the only one. Since the goal of the test is to screen for basic competence, it ought not to filter out people who could answer the question perfectly fine if it were being asked clearly, but who also perceive corner cases and ambiguities. Such a skill should make you more likely to pass such a test, not less. Therefore the question ought to say something like "What does the following code snippet write to the console?"
As you say, any competent programmer could solve this in a few seconds, if they are proficient, or minutes if they are not.
In my opinion, they should not put a count down because they are interrupting the programmer’s flow. If they really want to measure time, they do it without interrupting the programmer, hide the timer, let them take as much time as they want/need to solve these problems. Then, at the end of the quiz, show them how long it took them to give a proper solution, and take in consideration that time to score them.
If Alice solved 10 problems in 10 minutes, and Bob solved the same problems in 15 minutes, maybe there is something there that is worth highlighting in their profiles. Maybe Alice is able to analyze this type of questions much faster than Bob, and if that matters to the recruiters or potential employers, then allow the candidate to use that in their favor.
However, if Bob actually knew the answer, but they ran out of time to select an option, that seems unfair.
Depending on the language and how it passes in values, you'll have different answers. Assuming it's javascript the answer is 0,1. In other languages it might be 1, 1.
That's a tricky question, and if you're not good enough to understand why they're asking that then you fail!
This particular example is 0, [1], which is easily seen by pasting it in the console.
There may be some edge cases I'm not covering here, but JavaScript passes primitives by value (strings, numbers, etc) and non-primitives (objects, arrays, etc) by passing a copy of the reference.
What this means is if you reassign the reference inside of the method, it will only affect that scope, because the reference itself is a copy. If you modify the properties of the non-primitive, it will be modified, because the copy of the reference points to the same non-primitive.
Since primitives are passed by value then any modifications are not reflected outside of the method.
To the other poster below, in JavaScript, const with an array (object, etc) simply prevents reassignment of the variable, the array can still be modified.
This is more about knowing what const does to an array. Which is, IMHO, not how I would expect it to behave. So it's somewhat of a trick question. But I don't write too much JS so I don't know how well known that quirk would be.
a and b are shadowed in the foo function so the function only acts on the shadowed (and copied) variables. Once you return from foo, the const a and b are unchanged.
Not a great question as a pass or fail test imho because if you use shadowed names often enough to be able to parse this code in your head, you’re writing bad code. There is a reason why shadowing names is a bad practice: it’s hard to figure out what the end result is and account for side effects! But as a discussion on all the points above then it would teach the recruiter something about what the candidate knows rather than using this as a trivia question.
The const on b only applies to the object reference to b, and doesn't prevent foo from changing the contents of b. No doubt that's why the test question includes an array in the first place.
At least I assume that's how const works; I've not used const in JS.
That’s possible. In reality, if I ever came across that piece of code and had a bug to fix in it, the first thing I’d do is rename a and b in one scope or the other because it’s just asking for trouble otherwise and it’s not easy to understand what happens to a and b. That’s why I think using this question as a trivia question is bad. But the discussion we’re having about it clearly shows we both know what to look out for, that we’re competent, and is way more interesting than running JS code in our heads. Even if either of us gets the answer wrong, it doesn’t really matter as much as how we got there in my opinion.
That said, I think a hiring process that puts engineers in control (no ghosting, get data on when a recruiter looks at your application, search ranking by whether companies lie to candidates) is something that should exist. I want to try to build it!
However, I dropped out completely because TripleByte wouldn't let me see companies without entering in a desired comp number. My recruiter at triplebyte said I could just put in "$1 or $1,000,000" to get past it, but that just feels like the same kind of corporate HR bullshit that people have complained about for years. Just Google "recruiter won't proceed without desired salary" and you'll understand how many folks dislike that question.
The experience made me feel that Triplebyte wasn't interested in putting me in control of the process at all if they have such a user hostile requirement and require me to work around it by putting in fake numbers. I understand that many people will have no problem providing this info, but I prefer to see what companies offer me because it tells me how they value me and the position they're hiring for.
Almost all recruiters have been fine with that, but a couple haven't. To such people I wish a good day and go my own way. There are plenty of other recruiters in the sea.
I used to be much more relaxed about this, but if you get burned doing a few rounds of interviews and it was a waste you wise up pretty quickly - teams do not like time wasted.
I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons.
We're happy to talk through the different positions, levels, progressions and comp ranges for them all. But you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward.
You say "I've also found that folks who can't / won't talk money - usually not great hires for a bunch of reasons."
The same goes in my experience for companies which insist on talking salary before the job offer. It's a red flag.
"you have to have a clue about what type of comp would work for you (early on in initial call or two) or it's just not worth continuing forward"
Of course I have a clue, but I'm not going to put myself at a disadvantage in the negotiation by revealing it before I've been given a firm job offer and heard a number from the company first.
Also, in my experience recruiters are the ones who push hardest to hear salary expectations up front. They know it'll give them a leg up in negotiations, and some of them are even paid specifically to get that number (as one recruiter was brazen enough to reveal to me).
That’s because it benefits you as an employer.
I’ve been on the hiring side and I know there’s a budget. If someone is too high you want to push them out right away.
My flatmate in Singapore about 6 years ago was looking for a new job. Her salary was about 2200/m and she said the recruiter kept asking about salary.
I told her not to discuss it. She doesn’t /need/ a new job. If the company really values her then they will give her an offer. They have a budget. They offered her around 4600/m.
If she told them she was on 2200 it’s guaranteed they would have given her only a small bump.
Companies should never ask for salary info. Because it’s only used to get cheap labour.
Your clients know their budgets. Why not just be upfront about that to the candidates. Tell us your budget or reasonable expected range your client can pay. Then if (when) we find that it's well below what some other hungry companies are paying, we can save you and us time by passing.
If you know your client is paying low, then just be open about it. Something like, "Ok, this client is a bit below market (and tell the upper limit they will pay), but they have these good things to offer." If the client has no good things to offer, and the pay is low, then you're still likely to not seal the deal with a candidate no matter what. So just cut the game and get to the point.
If you're happy to provide a range, great, if I'm still there doesn't that generally indicate that the range is in line with my expectations?
I guess your comment kind of makes sense, I can see how corporate might interpret 'this person looks out for themselves' as a potentially bad hire.
This makes me think that all jobs position should have rather narrow target ranges which are stated up-front. Then you can discuss if you are too high level/low level for that job and range.
In this case, obviously a job seeker does have many possible employers to apply to.
So no, AFAICS you are wrong and the original thesis of "the company holds the advantage" is correct.
___
[1]: Consider the connotations of those words.
That time would not be "wasted" if the company had done due diligence on what the position and candidate were "worth" and revealed their target (or target range) to the candidate.
The real reason, I'm sure, is that companies think they hold the upper hand in the negotiation. And a lot of time with a lot of candidates that's true. It's clearly _not_ true for the class of candidate who's realistically got a $300k+ target. You are totally not going to successfully headhunt anyone in the top 1% or 5% of FAANG engineers if you try to dick them around with recruiting power imbalance power plays. _They_ don't need your bullshit. They will totally walk away from whatever time you've "wasted" so far trying to recruit them. You need them _way_ more than they need you.
(And I'm pretty sure that generalises a lot further down the experience/renumeration food chain than most devs realise. Fresh grads or devs with only a few years of experience might need to play the stupid recruiter games, but most companies trying to fill a mid level or senior role need the right candidates way more than those candidates need any one specific job. You can and should be able and willing to walk away from any potential job offer over stupid games like this...)
1 - I get real-world interview practice, and that always helps me in future interviews. Many companies ask the same or similar questions, and the more I'll be ready for them being asked again. In addition, when I'm surprised by a question I'll research it thoroughly when I get home and won't be surprised by that question again.
2 - I get a bit of an inside peek in to how the companies I interview at work. That's a privilege not afforded to many outsiders, and how some of these companies (some of which are world class) can and has helped me at other companies.
3 - A company might bring me in to interview for one position, but after the interview might determine I'm better suited for another. If I didn't interview with them they'd never know.
4 - As I interview with people I get a chance to make a good impression on them, so even if they don't wind up hiring me for that position they might want me when a different position opens up (whether at that company or another company the people I interviewed with moved to).
5 - By letting companies make the first offer I myself get a good salary survey of the minimum what all the companies I interview with are willing to pay for someone with my qualifications in that position. Of course I try to do my own research ahead of time, but getting salary numbers without getting a job offer is not always possible or reliable.
6 - If I blurt out a number first I'll it may put me at a disadvantage in the negotiation, as they'll just start the negotiation from what I say even if they would have been happy to have paid way more.
I hear this a lot, but how much interview practice do you really need? After the third or fourth set of interviews, they start to really blur together for me.
None of your points are false, but if you're casting a wide net and taking every interview without any idea if the company can even afford you, that's a lot of hours spent. The last time I looked for a job, I interviewed at only four companies and spent an average of probably 5-6 hours on each, and it was exhausting. I can't imagine making a habit out of it while also working full-time.
But, in my case, doing many interviews helped me understand what negative signals I am emitting during interviews [according to the interviewers]. There's nothing like an outsider's perspective, even if you disagree with it.
---
Example: in one of my last interviews I was told that I am too "cocky" and "over-confident" because when I was given real-time modifications to a homework assignment (that I completed before) I was smiling and saying "oh, that's easy!" and proceeded to make the changes live in my editor -- while sharing screen.
This feedback left a very WTF feeling in me. Dude, I am (a) enthusiastic about the task and (b) have enough experience to show you, in real time, that I can modify the code right there and then to suit the new requirements, and (c) chatting with the interviewing team while doing it... and all you can gleam from that is that I am overconfident and cocky? Seriously, get a grip.
For the record, I strongly disagree with their take. But it gave me a very interesting perspective -- namely to scan for the more introverted / likely-insecure people in the interviewing dev team and try and act a little more modestly to not rub them the wrong way.
It's a fact of life that most people suck at interviewing -- as I hope that I demonstrated here. One of the guys in that team simply dislikes certain kinds of people and he lets this go against his better professional judgement about my abilities. Not cool, right? Especially when you also take into account that the CTO who also attended all interviews agreed that my tech acumen is top-notch and they seriously couldn't catch me with my pants down no matter what they did.
So again, this helps me gather experience and culture and makes me more adaptive in the future.
Does it hurt your ego? Absolutely. Does it shake your preconceived notions? Much more than one would think beforehand. Is it exhausting? Hell yes, sometimes after an interview I didn't want to do another one for a week.
But, is it also very valuable to help you improve your interviewing and negotiation skills? Yes!
That's why I did lot of interviews the last time around. And I'd do it again.
The nice thing about the explosion of remote work due to COVID is that engineering salaries are beginning to normalize across the US, and there is now a lot more salary data available online, so it's easier than ever to see what you're "worth" on the market on average. This at least gives you an anchor when having these conversations, so you're not going in blind, and gives you a lot more leverage if they low-ball you.
[0]: Yes, I'm aware of the drawbacks of self-reported data, but it's still broadly useful in aggregate.
[1]: I'm making up numbers, but something like $120k - $200k, with the lower number being the minimum you'd realistically take. Usually recruiters are honest up front if they can't even afford your absolute minimum, but it still leaves you with a lot of negotiation room once the offer comes in.
Why even give an upper limit? It's not like you'd refuse $400k if it was offered to you, would you?
Most companies will naturally start negotiation near the lower end of your range, even if they would have started way higher had you let them make the offer first. I really don't see an upside to this strategy.
Because if you just give one number, even if that's your absolute minimum, the entire salary conversation is going to be anchored around that one number, and you're going to have a harder time asking for 50+% more than that. If you give a broad range, it's easier to justify countering with something closer to the top of that range.
> It's not like you'd refuse $400k if it was offered to you, would you?
Of course not, but nobody has ever offered me that. :) In all seriousness, yes, you run the risk of your range coming in below what the company would be willing to pay, but in my experience (and those of friends and acquaintances), outside of FAANG, I have found that to be extremely rare if your range is wide enough. The companies that pay super well are generally big, and thus well-known to pay that much, and I wouldn't give them any salary numbers at all because I'll have a pretty good idea of what they'll pay me with a bit of research prior to interviewing.
Of course, anecdote != data, so YMMV and all that. This is just my experience interviewing at mostly smaller companies.
Luckily levels.fyi has fairly accurate data for those same companies.
Agreed though in practice that just moves the pre-interview conversation from a comp discussion to a leveling discussion.
It seems to be terrible waste of time going through interview process to learn that they won't even come close to what you want. I would not do that to myself.
at senior levels especially, you should know what you want. comp bands are informed by huge data troves now and any edge you may have gotten before is gone due to the fixed and tight bands. just state your requirements and insist on “turnabout is fair play” ie they need to tell you the band. do this during the prescreen and save everyone’s time. if you kill it in the interview you still easily can negotiate to top of band.
If your desired comp number actually is $1M/year, will that just flag you as a joke to Triplebyte companies? With post-pandemic stock price increases a lot of engineers at FAANGs are making more than that now.
For any reasonable definition of "a lot", no they absolutely are not. A lot of people with 2+ years of experience who've either been promoted once and received a strong annual review, or switched jobs as a mid-level industry hire are making in the 200s and 300s, a few pushing into the 400s. Much more than that and you're looking at the top 1% (or less) of engineers at the top companies. Not a lot.
Source: was a mid level engineer at a FAANG until I left late last year, knew seniors and principals and their comp. Nobody was making $1M/year.
Nobody that makes 2x what their coworkers do at the same job/level casually discloses their comp at the water cooler.
A) your company is engaged in disturbing monopolism; B) your company is engaged in disturbing labor exploitation and arbitrage; or C) your company is engaged in disturbing environmental destruction.
Would love to be proven wrong. Am I?
Edit: option D) is that American cardiologists are not as exclusive and skilled a labor force as I would prefer to believe ;)
Also, staff level is a tiny slice of the engineers at FAANG. So still not “a lot” even if your stock appreciation numbers were correct.
They’re all doing well, but even Amazon hasn’t doubled their stock since prepandemic.
There are definitely more people making 1m/yr at FAANGs than their were last year, but I agree with others that it’s not “a lot.”
AMZN in May 2019 = $1775, today = $3496, 96% appreciation.
AAPL in May 2019 = $43.77, today = $130.87, 198% appreciation.
GOOG in May 2019 = $1103, today = $2508, 127% appreciation.
That's all over the last 2 years. Market low was apparently Dec 2018, before the Fed started cutting interest rates, so the numbers are a bit bigger since then. They were pretty steady through 2017-2019, so anyone who got grants or refreshers during those time periods has benefitted from that appreciation.
It's incredible how mileages vary. I recently had to strike companies off my list which demanded to know my last salary.
I offered them my salary expectations, but they would absolutely not accept that, and preferred to take the fact that I would not tell them the exact salary at my last position as a reason to decline to immediately stop the interview (with half an hour left in the call).
This wasn't a one-off either, I would say about 5% of companies (biased sample) try and get away with shit like this.
(This was German-speaking Europe)
In which case the correct answer is "Das geht Sie überhaupt nichts an."
I don't think this part is off. Why I never went to Triplebyte and will not go, I think doing tests is dumb and don't wanna do it. Like original commenter said, you can't change model that quickly.
There is a large pool of people who think that they are programmers, but are not. This is exactly what the infamous fizzbuzz test was intended to weed out. I personally know a top notch recruiter who credits her success due to learning early in her career how to filter most of the junk out. For example a "Perl programmer" who listed Microsoft Word as a skill was probably not worth looking at. (Note, this is an example from 20 years ago. But the principle remains.)
If you create an engineer focused site and DON'T find a way to segment engineers by ability, you'll find that your negotiating power is based on the average of your candidate pool. Which means that you'll discover why traditional recruiters kowtow to companies and not the other way around.
The problem is we're all spending weeks and weeks on weeding out fakers. We've moved to a quick pop quiz right after we get an apply. This takes out 85% of the candidates (it makes me sad when I look at how simple the stuff is that weeds them out...). Two interviews are more than enough to land on a hire after that.
Our company is a conversational recruiting software company (you apply, we say hi, and screen candidates in real time and then hook the candidates up with an interview). We started doing the screening in our chat flow and it works really well for us.
& maybe I'm wrong but I thought the entire value proposition of TB was that their pre screening and testing process made the whole process much less noisy for your hiring managers. If you still end up needing technical interviews and aren't guaranteed that all the candidates you get have gone through some technical screening, what's the point?
> Going forward, our assessments are a purely optional means by which engineers can show their skills to employers (who overwhelmingly tell us that they trust our scoring), rather than a requirement for entry.
I think the idea is that an employer can still filter out the unfizzbuzzables, but has the option to ignore that for candidates whose stellar track record is a stronger signal.
I'm a little bit skeptical, because i think there are lots of people with apparently strong CVs who are still weak when you actually put them to the test. But maybe not at the very top end; i am really thinking about people with experience in various investment banks, which is perhaps less of a signal than Apple then Netflix or something.
The trick here is you need to figure out a business model where you don't take money from employers, because once you do its just the long slide to becoming a shittier LinkedIn.
Colorado now requires job postings to include something more or less like a "reasonable salary range". It hasn't been perfect, for example there are some companies who now just restrict their online job postings to say "except Colorado" instead of adding a salary.
But as someone living in Colorado it's been nice to know the salary range up-front more often and to have confidence that I can ask them for a salary range without getting the question turned back on me: "Well, what are you looking for?"
(c) An employer, upon reasonable request, shall provide the pay scale for a position to an applicant applying for employment.
[1] https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml...
So it seems it’s not enforced.
If you have a janitor in CO, every job is an improvement and potential promotion. Having to first go to that janitor for every position is going to wear out FAST. Posting for the CTO position? Go to janitor first. Posting for CEO position - same thing - that's what is going to drive folks away.
The biggest strength for me is that I can ask them and they have to tell me. When I was job searching before I'd ask the recruiter and then they'd always ask me what my target is. Not showing your cards is like negotiation 101. Plus it's made fielding messages and emails easier. If they provide a range, I can just easily say "This position sounds interesting but unfortunately the salary range isn't what I'm looking for currently."
You're right that a lot of positions aren't great about this, I've literally seen "Pay starting at $0.01" or ranges literally $30k to +$200k. But I've seen it as a benefit in the negotiation process and it's just been easier to field recruiters.
> total comp
This law only covers salary as far as I know, so equity and such aren't included. Total comp is different.
It's not perfect, but I feel like it's wildly helpful for a ton of people. It helps transparency for a lot of positions and I feel like it rewards honest companies. If equity and bonuses are a big part I think it reduces the overhead for talking about salary to clear the way to talk more about equity and bonuses.
It's also in the early stages and CO is the only state with a law that requires disclosure up-front to my knowledge (someone mentioned CA has a law, but they only have to answer if asked). It feels like a fantastic first-step to me.
Unfortunately it looks like companies can just state wide ranges (which don't help applicants understand the market any better) and they can also omit other compensation elements beyond the base salary component (stocks, cash bonuses). That makes the disclosed information not very useful.
I personally like this proposal and am probably going to at least draft a hypothetical spec of it and see how it fleshes out. Thanks for the suggestion!
Pretty obvious value-add for job searchers. Not to mention that having access to this data would enable tons of other product features. Shows that most services/sites don't care that much about the applicant experience.
https://news.ycombinator.com/item?id=27256776
As CEO, you:
* Grossly expanded profiles without user permission.
* Laid off tons of staff right before the COVID lockdowns. (What has been your personal returns since April 1, 2020 by the way?)
* You have been privy to discrimination happening in client company on-sites yet did nothing to the client companies.
You’re a CFO, an investor. You’re not a CEO, and you certainly do not have my trust as a developer.
Sure, having metrics on how often a company ships code, etc. would be helpful, but frankly I have about 10 companies on my list that I'd work for, and I doubt any of them would give you this data.
What I'd really like is not having to practice leet code problems for 3 months before I interview. The older I get, the less motivated I am to do this. I've worked in senior engineering roles at big tech (FANG) with 3+ years tenure. I don't think I managed to not be able to do my job and fooled people for that long.
I've interviewed people I KNOW could do the job, but they didn't get the optimal solution, or were nervous, or N things in the loop, and we ended up not hiring them. There must be some incentive to create a system that lowers these false negatives.
I simply do not care about things like, “what does malloc return”. I do not care what people know about bloom filters in Postgres. The 99.99% of web developers don’t have to know these things. I do not care if you can implement Tic-Tac-Toe.
It’s poor interviewing, which is the entire product they’ve been offering.
I’m generally quite pro immigration but when BigTech goes to DC and bellyaches about how they need higher quotas because they can’t find enough workers it really sticks in my craw.
Reminds me a bit of the classic example of chutzpah—-the boy who killed his parents, then threw himself on the mercy of the courts, asking for leniency because he was an orphan.
This is a space that unions fill well compared to for-profit recruiting agencies. The organizations give engineers control and leverage in hiring, and can help them up-negotiate. They also allow employers to hire from pools of skilled talent.
Whereas with recruiting agencies, the customers are employers, not the candidates. Incentives are not aligned to give candidates control.
People at the recruiting agency usually get paid a commission as a percentage of the first year of the hiree’s salary (not taken from your paycheck, but just in general). Which means, it is in their interest to get you as high of a comp as possible. Which also means that I dont have to bs around with the recruiter about my salary expectations or anything like that. I give them the upper range of what i want, they tell me either “sounds good” or “they are easily willing to pay more, so why not try $X instead” or “they cannot pay that much, but there is this other company that we work with that can offer you as much.”
And if you work with one recruiter consistently, it gets even easier, because now that the recruiter knows my total comp expectations, she pre-filters opportunities for me and only contacts me if there is a position she is aware of that pays at least as much as I want or more.
Of course there are downsides too, for example, the limited choice of companies that the recruiting agency is working with at any given time, so i still have to find some opportunities and interview with those companies on my own. But as a supplementary option, working with a good recruiting agency has been quite nice for me.
P.S. The type of a recruiting agency i am talking about is the one that is focused on a specific sector (in my case, fintech/quant), so it might be a completely different story for a “generic software dev” recruiting agency.
Beware: this is not universally true. What you typically see is that it leads to closing as many candidates as possible. A recruiter can make more by closing 2 candidates at 80% of max comp than 1 candidate at 100% in the same time.
This is a well known behavior in real estate sales, which have a similar commission structure.
I am saying that because I’ve been working with her for at least 3 years now, and I am yet to take a single offer from the companies she set me up with (either due to them getting cold feet last second or simply due to me not passing some of those interviews or me declining due to some red flags with the companies). If it was all just about high volume for her, it would have been in her interest to drop involvement with me after the 3rd unsuccessful attempt to set me up with a fintech company.
Even worse, real estate pays both sides so realtors are heavily incentivized to get their buyers to purchase from their own sellers so they can get both sides of the commission. Instead of paying them 3% to act on your behalf and do a bunch of leg work they get double and now they can do less work and aren't strictly working for your benefit.
Working on commission just leads to wanting to make a deal happen. Even ignoring the time aspect of it, if a recruiter has much better odds of making a placement at 80% max comp compared to 100% max comp, the 80% is still objectively better. Why risk not closing the deal when the expected return is negative?
Collective bargaining is inherently about eliminating competition between workers. That is, it creates a labour monopoly to balance out a labour monopsony. That might (might!) be a good thing for autoworkers. It would be an unmitigated disaster for software engineers, and the tech sector in general.
People who write code are unbelievably productive compared to just about any other job function, and software engineers (who we'll define as people who write code for tech companies) are even more so. Part of eliminating competition between workers is making sure that nobody works harder than anyone else. That would be devastating to the productivity of software engineers, which would make them less valuable to tech companies. That would lead to a decline in salaries, and general stagnation of tech.
Another aspect of preventing competition is seniority. One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top. A union would tell that kid, "sorry, you'll have to pay your dues as a junior engineer for 5 more years, then you'll be eligible for promotion. In the meantime, we'll give that job to cwp, he's got the required seniority."
Just think how this would affect tech companies. Sorry, you can't pay more to attract the best. If you're really careful about interviewing, you might be able to find some people with both skill and seniority, but it'll be tough because smart, ambitious people have all gone to Wall St or whatever industry has the best opportunities.
A yet every movie star in the US is in the same union as B-movie and infomercial actors.
> Another aspect of preventing competition is seniority.
Weird, considering pro-athlete unions regularly help kids out of college quickly become multi-millionaires.
> One of the archetypes of the tech world is the wunderkind—the brilliant kid fresh out of school that rockets to the top.
Weird, considering unions have been at the forefront of education, apprenticeships and training that helps kids out of school rocket to the top, the same apprenticeships and training that companies refuse to fund themselves.
> Sorry, you can't pay more to attract the best
Weird, considering SAG-AFTRA regularly helps movie stars up-negotiate multi-million dollar contracts.
But the transparency you need to extract is mainly financial, that’s the big asymmetry. I think people care more about money than they say they do, and less about release cadence or test coverage or whatever than they say they do.
So triplebyte should be called “get me a raise.com”. I tell you what I’m making, smash all your tests, and you find me somebody who wants to offer me e.g $50k more or $100k more, and that WILL be the offer I get. And if the company reneges on that offer then it’s flagged to warn others, and if they do well by their employees they are rewarded as well.
That said, the idea of "we will offer X to anyone with Y quiz scores" as a showing a company can make is pretty cool, and is definitely in a "company makes public conditional guarantees" space that we're interested in exploring.
Does this guy have a patreon?
I am looking forward to progress toward a hiring process that puts engineers in control, such as by allowing them to accurately record their current role in their profile.
In 2015, candidates with the same schooling scored about 20 percentage points lower than 2020, the last candidates I used the test for. The number of questions doubled, and then the test got much easier, by eliminating more challenging programming questions and replacing them with questions with giveaway context. By comparison, according to my data, about N=43, until about 2018 being a senior versus a junior in college CS programs is worth about 10 percentage points on the test; going to Harvard instead of Berkeley is also worth about 10 percentage points. In 2020, the last tests had no predictive features.
I stopped using the test, because it became too easy and too noisy to be informative.
I recognize some of the coded language in the blog post. There are definitely more lucrative opportunities in recruiting for DEI. I don't know if it will last. If you're still jittery about the public-profile-by-default thing, which by the way, was totally irrelevant and overblown IMO, this may not be a pivot for you.
One thing I see in the data is that at MIT, women and men performed the same, controlling for seniority. This wasn't true at the 3 other universities that produced enough data to measure.
That said, what really is the best way to hire candidates? I'm not convinced having binders full of engineers is special, there are almost always more candidates than jobs, at least 5:1, in every non-credentialed industry vertical. Anyone who has worked at a jobs (or indeed any matching platform, like the Common App or Tinder) knows that.
Then there's this long thing about asymmetries or whatever, warble garble about missing information... It has never, not once been my experience that someone seeking a subordinate role at a typical private company with preferences like "pair programming" or whatever have ever been better than someone with no preferences at all.
Maybe it helps to engage in the vanity of whatever trendy workplace trend is hot for whatever vertical. But like, if you're being intellectually honest, if you thought pair programming was important, you'd pair program at TripleByte, but you don't, you know in your heart of hearts none of that shit matters, so why are you putting stuff like that into your search system?
Indeed and ZipRecruiter are ad arb companies. They don't care. Private universities lead, not lag, DEI at giant companies, so it's hard to see how to compete against them in that core business. It will still come down to a real defensible opinion.
Do you have more valuable inventory than ZipRecruiter for DEI candidates? Who knows. What an uninteresting question. Apple also hires people who just make shit up on their resumes, I know two - though they weren't engineers.
If this were true for "qualified" candidates, there wouldn't be a labor shortage, right? ;-)
They told me the actual number of qualified candidates was usually just a handful or less.
I doubt this is true in SV, but I don't live in SV.
That's interesting to me. Most reputable rankings put Berkeley at #2 or #3, and Harvard is usually around #7 or #8 (and US News puts them at #16!). I wonder what the cause of that discrepancy is.
From what I can tell, anyone who is a Harvard student can declare the CS major if they pass the pre-reqs. At Berkeley, passing the pre-reqs doesn't get you into the major -- you still have to apply for a limited number of spots, which generally means having a 3.8 GPA or better in your pre-reqs. So effectively both programs have the same acceptance rate.
Berkeley EECS is sub-7%.
Most clinical doctors have social skills and crystalized intelligence from domain expertise, but aren't typically mistakable for particle physicists.
People don't necessarily require the proper sheepskins to possess fluid, crystallized, and/or other domains of intelligence AND the skills, personality, and experience relevant to excelling at a particular STEM role.
Pedigree is mostly used for social filtering and business leadership board packing, but if someone wanted to create an elite monoculture of staff, lacking in cognitive and personality diversity, by all means, go right ahead.
PS. I. Don't even get me started about a car full of CS IITians talking about the JEE and high-placers who seem normal on the way to snow country for snowboarding and gambling.
PS. II. I bombed an Apple interview for a mid-career role by being too intelligent and too maverick compared to the group of compliant, I hate to say, yuppies. It was an interview panel of about 9 people and they were just speechless. The other time I bombed an interview for being too smart was about 10 years before that when I was 20 at the old Borders bookstore in Palo Alto. Moral of the story: it's important to play dumb where appropriate because most people are relative-intelligence insecure.
PS. III. Sorry, reader, for the rambling and discontinuous thoughts. Absurd endocrine values of unknown etiology currently... doctor appointments pending. ):
Are you sure they were speechless due to your high intelligence and maverick qualities?
That sounds awfully specific.
how do I reply to recruiter contacts? applied to a job, received a request for interview from company. Nowhere to reply on the platform? How do I get in contact with them?
Should there not be a reply button right there with recruiter's message?
p.s. I already emailed support but, my past experience tells me they won't reply.
To be clear, I'm only addressing the comment about how people thinking highly of them are gullible. I had a great experience, twice, within a few months, and that included going through a time of huge uncertainty.
That being said, I was not happy to see each of the changes they've made over the last year or so, and this does not strike me a good direction for engineers. I suppose I have to agree that they are not doing this from a position of market leadership. Perhaps they were providing asymmetric value to engineers, and at the end of the day the engineers are not the ones paying for the service.
This is a very good insight. Unless candidates pay more than companies, the incentives are off.
> We got jobs for over 1000 engineers
Given how many years Triplebyte has been running, "over 1000" seems surprisingly low. I wish them luck with their pivot.
For sure. That's very likely why they're making this pivot. Exclusivity was their goal, but it just turned out to not be a very profitable goal for them, it seems. I was basically just saying it seemed to be a "theory" issue rather than an "implementation/execution" issue.
dev)
> I forgot how to implement the zig operation in a splay tree.
employee at Triplebyte)
> To put some hard numbers to this: further down this thread, there's a post about how "any engineer" could answer a question the poster thinks is too easy. I looked up the question in our back end and, in fact, barely a third of people who take our quiz get it right (the correct answer isn't even the most common one!).
So there 1000 job matches and at most (("hundreds of thousands of engineers on our platform" / 3) - 1000) who were incentivized to answer these contrived problems, did so correctly and still couldn't be matched.
So even among the large pool of engineers who have gone thru the process, met some arbitrary threshold of engineer-ness, there's still a huge mismatch between corporate/prospective employee expectations, that I'm not sure will be able to be overcome quickly even with these new initiatives, but it's interesting that they are being pursed now (not surprisingly after the shift in working environment after massive government restrictions on freedom uber alles).
Yeah, different companies want different things and are ok with different things. My best experiences have been with companies that have taken less than 2 weeks to hire me with no testing whatsoever: just a couple of calls. Other companies want and only select for the contrived coding tests and programming jeopardy, and that's fine, but I want nothing to do with them at all.
I would be interested in using triplebyte if they explored something like this: https://news.ycombinator.com/item?id=27545446
Maybe even allowing 3rd parties to bet on specific hiring decisions (in addition to what is described in the link) could add another layer of incentivized feedback loops to the process (esp if the people making the hire/fire before 3 months are made known) and add another layer of market driven signals for them to pay attentions to. Maybe make it more expensive to chase certain types that are in higher demand, and less expensive to chase others types that are in low demand (not the salary to the employee, just the payment for filtering). Maybe less expensive for shorter refund windows, and more expensive for longer refund windows.
$25k*10k = $25M / 10 years = $2.5M / year. That is peanuts compared to how many people they employ. Plus the ads they run...
One. The problem domain they are in is on its way out, and they've decided to defect instead of being the ones left holding the bag.
Two. They hate what they do for a living.
The first company to stop making buggy whips probably did not do it because they weren't selling any. The fact that they ceded market share to their top competitors probably sunk the barb in deeper for them: Yeah people are buying cars but our sales numbers are still going up so why should we change?
In tech we see companies all the time end up being complicit in the destruction of their own industry by reducing its relevance in some manner or other, and by exiting early they have more options of destination. If I jump to the same industry as a competitor a year later I just look like a copycat. While some copycats copy other people, only better, they are the exception to the rule. Most churn out uninspiring derivative work, and if they can't at least do it cheaper then they end up on the scrapheap of history.
At that point, you can either keep going with what you're doing (and run into an inevitable wall later on) or you can foresee the future problems and pivot before they become unmanageable (which is what we did). Relatively few startups find their final model from minute one.
Funny but whenever I think of Triplebyte, all I remember is seeing their ads everywhere. I am personally a fan of Indeed and they do a good job sending leads and applicants other than good old Linkedin.
When I was fresh out of university, I had trouble finding a job. I had worked part-time at a small company through the last few semesters, but that company blew up just before I graduated. Every interviewer I ever met with was impressed because I really knew my stuff, but had trouble getting in the door due to an unimpressive CV and awkward mannerisms.
I did very well at Triplebyte, scoring in their top bucket and getting top marks in several of the sub-categories. Those metrics are relative to all engineers, which would put me even higher amongst recent graduates. I also did very well on every software project I worked on and scored top marks on several other objective tests, like the Major Field Test.
I was exactly the sort of candidate their system should have been able to help. I never even got an email out of it, let alone an interview.
Waste of time.
Fortunately, most of the dirt thrown at recruiting teams is wrong in the first place. I don't know any hiring managers who make lowball offers that have a low chance of acceptance - the costs of the process, the risks of hiring someone who starts mad at you, and the value of winning someone who is actually worth hiring, are all way more important than some incremental cash saved.
Well, presumably the idea is that engineers, especially high quality ones, will be drawn to the recruiting platform that has more pro-engineer policies. And so if you avoid using it, you lose access to that subset of engineers.
It's still a chicken and egg problem, though, because engineers have little incentive to join until there's enough companies.
However, I think that even if they build an attractive, differentiated experience, the advantage is unstable. The problem is that "traditionally strong" applicants (those who already have positive ROI from existing processes) are disadvantaged by being exclusive to one platform. So, from a hiring perspective, TripleByte's differentiation in terms of unique resumes is going to be from people who are harder to assess.
I'm actually fine with that as a value case - we've had great success hiring from bootcamps. "We help you reach strong nontraditional applicants" is a great value proposition. But it's not the same as a position of power to dictate the hiring process. Maybe a better pitch would be: we run a process that helps great non-traditional talent and great non-traditional companies find each other.
And as a HM, I was never interesting in using them, and we continue to use our own internal recruiters or local recruiters who know the market.
Having worked at a startup that was trying to disrupt the hiring/recruiting market back in 2010. I'm curious to how effective they were....if at all.
Edit: update, maybe I'm super skeptical because of all their cheesy ads that were all over Reddit for so long?
On related matters. I know several companies that have pivoted out of the "hiring/recruiting" space because apparently that space is not easy to scale/automate.
I guess they didn’t hit product-market fit with candidates. Two sided markets are tough.
That's worthwhile.
It’s possible that they just couldn’t code with someone watching but the effect was the same.
Note that I didn’t have control over the interview process, so I couldn’t switch to a take home or something else.
The resumes seemed like they had relevant experience. That’s why I pushed them forward to the tech screen.
My experience is different. I've been through the TB process three times, back in 2017 where they flew me to SF from Asia for Flexport, Mixpanel and two smaller startups, and then again this year where I went to the final stage with FB production engineering and TB itself. And in between as well i created yet another profile on the system after shelving prior ones for a fresh start. I have a non traditional background, (ie, an "outsider" developer, chem degree and self taught hobbyist programmer since young), i take my craft seriously, have a bunch of popular open source products and a successful low touch indie business. But i don't consider future success guaranteed so i like to keep talking to companies to find that great opportunity for me. I know my tech skills in some areas are not the best so i like to stay in the loop, stay up to date and relevant and stay in touch with what companies are looking for. By far the best hiring process experiences I've had have all been through the TB platform. And my "open rate" for applications and inbound interest on TB is way higher than any other methods I've tried (SO jobs, WWR, HN). The quality of companies, hiring process and level of interest I've received via TB has been epic and, at least in the past, even the TB support staff are super helpful going so far as to provide personalized feedback on profile improvements and resumes. I'm not an expert at getting hired but i couldn't have asked for anything more from them. They were fantastic. I only have great and wonderful memories of the time i spent in hiring processes of companies i "met" via TB. So this is my experience of it, and this is despite the fact that i have never ever received an offer from any company I have met on TB, including from TB itself. I don't blame them. I just consider my own skills are not at the right level, and the fit is not right for the companies I've met, while i work to improve myself and my skills. That's all. It's a simple and beautiful experience I've had with them. Soooo different to hiring through other platforms (where funnily enough i have recieved offers). I've met and talked to ammon and many other TB team members and they have all been cool, friendly, passionate people. From what I'm reading of this pivot, they're being vulnerable and open to learning, and using what they've learned so far to create something even better. I'm not gullible, I'm not saying they will succeed. But they're trying. And they're earnest and sincere. What more can you ask for? This has been my deep experience with them. I think the most vocal critics are talking merely theoretically without any experience of TB nor deep nor accurate understanding of who they are, what they do and what they stand for. To them i say, give it a try. Investigate before contempt! ;P ;) xx
this post is marketing with commenters associated with the company
I've been wishing for something like this for years. Especially for early-career engineers or people from non-traditional backgrounds, this is insanely valuable because it helps you to know avoid wasting time on applications that will never go anywhere; avoid typing in your resume, line by line, into another form after already submitting your pdf. Avoid writing a cover letter to a job that's already been filled. Avoid applying to a posting that's really just out there for a company to "gauge" interest, not for filling a real role.
If an application is rejected, fine. But getting a follow-up request 7 months after submitting a resume into a black hole is ridiculous, and I think any system that decreases the information asymmetry between the applicant and the employer that allows people to intelligently approach their career search is going to be tapping into something truly valuable.
Like you, we thought (correctly, as it turns out) that it would appeal to people early in their career. What surprised us is that very senior engineers also told us they loved it - we didn't think it was as much of a concern for them, but sometimes your users surprise you.
It's also a great example of how obvious low-hanging fruit gets missed until you start thinking strictly about "what can we do to make job seekers' lives better?". It's not a particularly innovative or difficult feature, and yet major job sites with a dozen times our engineering resources still haven't done it.
The last time I was a hiring manager, I committed to giving feedback to anyone I rejected after the very first screening step. This was about 25-30% of applicants, which (even for a role in considerably less demand than engineering jobs) ended up being many dozens of emails. It was a giant pain in the ass - and it's pretty emotionally draining because you're turning down people who would really, really benefit from getting hired. I did it anyway, because I'd promised to, but it was not fun.
Next time I look for a job, I will definitely be targeting both these companies.
I wouldn't charge for the service, however, unless building it as a freemium model.
/economist hat
I'm in this industry and doing well because of recruiter reaching out to me (multiple times) and not vice versa. The whole idea of applying for jobs just doesn't seem to work at all.
On the other end a recruiter reached out to me and I'll be doing the final interview with their VP next week.
I don't think we can draw conclusion at that stage (we are a dataset of 2..). But I'd love to have access to the LinkedIn dataset to figure out if applying for jobs is broken at scale.
It's ugly and mimicks real life... more effort gives better results but life isn't fair and never has been. Luck factors but always fight on multiple fronts.
My major are slightly mismatched though.
Then I stopped bothering when it's super obvious that isn't working. Particularly, on Linkedin you could see the number of people that has applied (maybe a paid feature, i don't remember now). And for any entry level software engineering and data science position it's always in the 100s.
The one recruiter that I had contact with did more work than all of these. I've since switched jobs (to one of the FAANG) and that was through recruiter too.
I also don't think field matters until one gets to PhD levels, which I did not.
Often enterprise HR is paranoid of honestly evaluating anyone for anything, to the point that many HR teams tell one another and engineering hiring managers that it’s “illegal” to assess candidate abilities on the way in the door, at all.
If you think about joining a dev team where no one checked if any candidate could FizzBuzz, you can imagine the workplace environment that can end up with.
Triplebyte is able to provide engineering managers with a stack of pre-vetted resumes so an enterprise can interview by its lowest common denominator HR policies, while still having a prayer a team will at least be made of candidates who can code.
The value of this to an enterprise stuck in this position is hard to overstate.
Hopefully along with expanding the talent pool per this post, Triplebyte can figure out a way to get well paid for this ability to help land actual coders on actual dev teams despite enterprise HR.
The need for this is huge.
// I see a comment below from a Triplebyte PM about score matching: https://news.ycombinator.com/item?id=27542621 … That would be slick: land candidates that genuinely raise the bar, but aren’t so far ahead as to be speaking a foreign language to the team. Of course, this would require also assessing the target team, and oops, we’re right back up against that HR policy…
There's an old quote: "Nobody ever got fired for buying IBM."
Large companies tend to be more risk-averse, so if there are two companies you can use for a service — $PromisingStartup and IBM — large companies will generally make a decision that's optimal to the specific decision-maker, not optimal to the company.
There are two options if you, as the buyer, choose either $PromisingStartup or IBM: success or failure.
If IBM or the $PromisingStartup succeed, then you've done your job.
If IBM fails, you can tell your manager "Who could've guessed! It's IBM!"
If $PromisingStartup fails, you'll have a harder time explaining your decision, and the fault will be with you.
The "Nobody ever got fired for choosing IBM" idea is useful to look at every decision large companies make, whether it's pivoting, choosing a SaaS product, or hiring.
TripleByte, in its current form, has been beneficial to candidates as well by giving enterprise employees a justifiable signal towards hiring them regardless of pedigree.
From the position of a candidate this sounds like a poorly run company I'd have no interest in working for. I wouldn't want to interview with them.
That sounds hilarious How could a dev team not be able to write 5 lines?
Its the same reason why a top notch, genious writer can turn out to be terrible at giving speeches, even if they're reading from a script.
Performing an interview is a different skill from writing code.
I think this is true for the day-long leetcode interviews, but to succeed as an engineer, even at the entry level, I think you need to be able to talk through how you would solve fizzbuzz. A big part of the job is communicating trade offs with other engineers.
I think if I could describe my personal experiences, I'd say that on a subconscious level, I know that how I speak and articulate details will be observed by the interviewer through a cultural lens more often than not. This may be unique to my personal background, so maybe this is different for other groups. But interviewers often get hung up on the use of different language/terminology, the types of jokes/jests that enter the conversation, how I respond to their language. And there's sometimes been an unwillingness to allow different terminology, perhaps because that's all the interviewer themselves knows and understands.
All that to say, a code interview doesn't always come down to the quality of the code.
I got shitlisted at one place for being the only dev with the mathematical insight to know that multiplying pesos by a dollars:yen ratio doesn't yield a correct conversion of pesos to yen. They let that one slide without fixing it, but the final straw that got me frogmarched was refactoring an if-else to a switch.
Such an eye-opening experience! Now that I’m on the other side of the table these days, I understand the importance of making interviewees feel comfortable and asking questions that allow them to start off strong.
They saw me as a god for fixing their shit and offered to hire me. No thanks!
Which turned out fine. I found a job where my pre-computer programming work experience was relevant, and here a couple jobs later I'm working heavily with and on parsers.
So. Count me as one engineer they rejected who doesn't hold that against them.
I'm pessimistic about this pivot though, for a couple reasons. One, it smells desperate. Recruiting is sales, and in sales, desperation is the kiss of death. I can't read that blog post without coming away with the strong sense that if this doesn't work, it's over for them. If I can see it, anyone can see it.
Two, companies aren't going to want to work with a firm that's allowing engineers to collect detailed information on douchey behaviour that the company might engage in. Does Triplebyte have the leverage and moxie to make them engage anyway? Probably not, see point one.
I wish them well, because recruiting is awful, interviewing is broken, and engineers deserve to have a better time of it given how demand-driven the market is, and likely will remain for quite some time.
Can't help thinking they'll be writing about their incredible journey within a year or two.
On the other hand, if a company does not engage in douchey behaviour, then using Triplebyte could be a way to signal that.
This fails a bit if companies can be credibly accused of douchey behaviour when they haven't been engaged in it. I don't know what Triplebyte's plan is here.
> Does Triplebyte have the leverage and moxie to make them engage anyway? Probably not, see point one.
Indeed. They labour the point that programmers have the upper hand in hiring, but that doesn't translate to Triplebyte having the upper hand!
I would think big tech companies wouldn't touch companies who might fold in the next 6 months with a 6 foot pole.
I have a total of 6 profile views. So you’re truly not missing much.
The most friction when switching jobs is the interview process -- and the question is why? I've been writing code for a decade+ now, working on startups, for known tech companies, public open-source, and have written a freakin' book! But, oh, my bad, I couldn't figure out a solution to your optimization question. I forgot how to implement the zig operation in a splay tree. I might simply not function well under pressure. Maybe I'm having a bad day.
Companies are missing out on literal geniuses by using outdated hiring practices. And, I get it, Google doesn't care. Amazon doesn't care. (There's an argument that they should.) They're huge and get X,XXX applicants daily. But why are small startups using the same hiring methodologies? It quite literally makes no sense. They're shooting themselves in the foot. I'm very passionate about this, and I'm working on a book on how to hire engineers. Maybe I'll actually finish it one of these days :)
Past companies and open source contributions are not perfect indicators. I've seen people with stellar resumes fail to even know the syntax for an 'if' statement in any language of their choosing. I've had people absolutely ace takehome exams, yet in person don't know how to write a function.
False positives are also much more expensive than false negatives, so companies would rather accidentally weed out good candidates than risk hiring bad ones.
If this is actually true, which I highly doubt (I've interviewed people, too), your process is terrible at preselection. I've heard so much about this "programmer charlatan" that lies on his resume and ends up blowing up million dollar systems, but have yet to see any tangible proof.
> Past companies and open source contributions are not perfect indicators.
This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
I've seen similar. I wouldn't say 80%, but somewhere in the 20% range of people on phone screens. If I ask a similar modulo question that isn't obviously a rephrased FizzBuzz (i.e. "write a function that returns every 4th item in a list"), that percentage goes up dramatically. A lot of candidates memorize FizzBuzz without actually understanding what's happening and how to use it.
> This is self-contradictory: how/why would any open source project let someone that doesn't know fizzbuzz contribute to their codebase? I run a tiny throwaway open source project (~600 GH stars) and even my reviews are pretty stringent. This point of view is not consistent. It's like saying "I know people that ran in marathons, but couldn't even run for half a mile in my interview."
There are no time constraints, and no way to tell how much of their contribution is from their own knowledge and how much is copied from StackOverflow/other open source projects/etc. The point of view is consistent; I can't run a marathon, but I can walk 26 miles. That's fine for the local charity marathon, but it's not going to cut it if I want to do competitive marathons. The constraints and expectations are wildly different.
I can run a marathon but I can't run 100. Your interview is one of many that I am making time over lunch. I'm tired and getting a perfect mark on a take home isn't as important as getting it done quickly so I can find time to apply to other positions before I go back to work.
If you are not paying someone to do the interview you are not going to get anyone's best. Even if you pay people are trying to juggle things to get the interview to work.
And to the parent post. 50% of people being late means they took the time out to talk to you and come to your offices and didn't gage the time or didn't think it mattered. That doesn't make them bad programmers and shouldn't even matter to the interviewer. Was the car service the employer used late picking the employee or did they have to come on their own?
Even ignoring the fact that you keep moving the goalposts and are taking my "marathon" analogy way too literally, the argument that any kind of real-world coding involves "time constraints" akin to a 30-minute stress-ridden whiteboard interview is just bonkers.
They could maybe just be making small or simple changes. Or perhaps mostly working on markup, templates, CSS, etc.
It's possible they know a bit of programming but just don't happen to know how to check if a number is evenly divisible by another number. This isn't something an open source project would necessarily know you don't know. (I agree it's something any programmer should know, but I can see how someone could contribute certain things to certain projects without that knowledge gap being discovered.)
I learned about the modulo operator very early on, but if I didn't know it existed, I'd admittedly struggle with the question for a bit and would seem like I don't know how to program; especially in a pressured interview environment. If it were Python or something I think I'd come up with something awkward like `if i / divisor == float(int(i / divisor))` within a minute or a few minutes, but who knows, maybe the pressure would mess with me.
I dunno about the back half (blowing up million dollar system) but the front half, certainly.
We used to ask "write min()" at a previous employ, and the pass rate was probably <50%. I still to this day structure my coding question around "it has to get them to write a for() loop". It's not trick questions (I understand why people don't like those, and those can die out, yes.) it's things like "here's a sample grammar, how would we go about parsing it?" which is something that I've used any number of times in my career.¹ Other candidates fail to display understanding, let alone deep understanding, about things like HTTP or Linux. Not exactly niche topics, and during this example period my employer at the time was clearly posting for "backend software engineer". And not just one or the other, but all of them, simultaneously. Since I can't establish any proof of anything, so… what other choice is there, but to pass?
¹while there are parsing libraries, yes, and I'll use those first / when I can, I would also say that 50% of the time, for whatever reason — better error reporting, better control — I end up going with a partially or mostly hand-rolled state machine or recursive descent parser. Also, while I wish people would just encode their data in something like JSON, or YAML, or whatever, it doesn't really matter, they. keep. making. new. grammars. Often inside those other grammars. Just today I had to deal with a config — a YAML config! — that wanted "key values". Turns out, it wanted strings formatted like "key:value", not YAML mappings.
With a bad pipeline, it's extremely likely, especially if you cheap out and try the bootcamp route (there's a reason Lambda is desperate enough they will "loan" a student you can try before you hire!).
It costs nothing to send a resume after all...
That's maybe a little unfair (plenty of people who aren't engineers take our quiz out of curiosity), but the basic point here - that companies are frustrated because they keep getting fizzbuzz-incapable applicants - is more-or-less accurate. That has been (and continues to be) a big part of our offering on the other side of our platform: in the same way that we can provide data about company behavior to engineers, we can provide independent verification of an applicant's skills to companies.
Wow, sounds like you either need to change your sourcing or improve your early screening. If that many poor candidates are making it to the interview stage something is wrong with the process.
Most either don't show up on time or fail fizzbuzz.
Of the people that turn up late to the interview, what percentage pass fizzbuzz?
And yet they will still email you, year after year… as a dev that's worked in finance/academic research labs and for low funded/bootstraped startups all over the world for the past 10 years… I'll never get into these big companies because I simply try to find companies that need work done yesterday, see my experience (maybe reach out to past corps i've done work for), and give me an offer for at least a short term contract (and these big companies will never do this).
The worst is when I've think I found an interesting company (that on more than one occasion, publicly likes to complain about not having enough devs… lol), start talking a bit and they send me a triplebyte link… maybe it will be different this time, but I doubt it because these behaviors are too ingrained in a lot of corporate processes (even if its mostly "big co does this, me small co must copy"-NPC type thinking)…
I have Github repo. I have my own portfolio. I have wrote flutter app deployed to play store to demonstrate I can function individually and still can write production ready code. And All the above just on my own spare time.
But every company I have applied told me my skill set doesn't aligned with them. I get response from them that there is no traction on my resume. While some friends get interview after interview and others don't.
I am not saying that I am the best but we need different process where entry is given and now show me what you can do.
I moved from engineering to management over 5 years ago now and still in interviews for positions like "head of" in 100+ people companies or other management roles in bigger companies I got asked the usual algorithm questions.
I just have zero interest to prepare for these interviews, I don't have the time or the motivation, and I honestly just don't care about the answers. Most of my job, and my team's job, has nothing to do with them.
The world might be different in FAANG, and admittedly I have not worked there (though I have worked for other top companies) – but most companies are not at that scale, and they should not use the same metrics, particularly for management roles.
* The blog post complains that even experienced engineers they interviewed couldn't write some simple code on paper. Did anyone stop and think that maybe it's writing code on paper in an interview setting that's the problem? Stop and think about this. Imagine writing code like you usually do in a text editor. And then writing code with pen and paper. Backspace? Can't do that. Copy/paste some line to move code? Can't do that. Add some newlines? Can't do that. Run the code using the compiler? Can't do that. Of course most developers, even experienced ones, will do poorly under these conditions, this is not how code is normally written. On top of the already stressful situation of an interview, there's this suddenly completely foreign and limited way to write code.
* Even the blog post only mentioned fizzbuzz as a quick, easy question to filter out zero coding ability. Another way to say this, it was meant to filter out coding skill <= 0. Not skill <= 50 or skill <= 90. Zero. After that, the interview was supposed to move on to things that actually matter for software engineering. The present day interview has strayed extremely far from this. These days, you get asked two leetcode questions during an interview and even if you perfectly answer the first one, if you don't get the second one then you're an automatic fail. That was certainly not the intent of the original fizzbuzz, since answering any leetcode clearly shows coding ability. And the interview never has a chance to move on to the part that actually discusses software engineering skills. Or even if it does and they do very well on those interviews, the failure of only getting 1/2 leetcode questions correct overrules everything else since they must avoid "false positives".
(N.B. I was being a bit hyperbolic, but, yes, for example, I highly doubt Steve Jobs could.) The spirit of my point is that most smart people don't care about the kind of idiotic minutia typical hiring panels ask. They care about interesting/creative problems.
Tongue in cheek, good luck on the pivot guys, it is a tough problem.
https://keithschwarz.com/interesting/code/?dir=find-duplicat...
Our principles at the time were very similar to what Triplebyte is trying to do now: give power back to developers.
We build a version of "hired" (did they die?) before "hired" even exist. A matching platform that was matching people with companies that was sharing both their desires and values. Small/big company. Consulting/Product company. How much $$ do you want? What technologies you're interested in? Where do you want to work? All of this anonymously of course.
It was all fine and dandy, the company actually still exists and generates revenue, but I'm overall very bearish on the recruiting space.
It's a sales game. It's a numbers game. And on top of this the people top companies REALLY want to hire aren't hanging out in these recruiting platforms. So as a platform you're only feeding average/average low candidates to companies, and in the meantime, companies that have in house or externalized sourcing resources get better results by going over linkedin and their employees network.
It then becomes a financial game for companies.
And platforms that are not linkeding rarely win.
Maybe I'm wrong here, but it gives me the impression that Triplebyte doesn't believe in it's ability to get candidates into jobs. If companies pay (a not small sum) when candidates actually get hired, then I feel that incentives are aligned. If it's an annual fee (with no option for contingency) then I wonder what incentive the company has to make sure what I care about (making a hire) is the thing they care about.
90% of the other major players (AngelList, Hired, etc) in the market offer this... so why not Triplebyte? Perhaps I'm missing something.
Then the company is incentivized to use you for junior hires. Maybe you'll get a few freeloaders who only want to hire juniors, but even those will eventually be seniors who love your platform, and at the same time, will boost your stats on the number of juniors you got jobs for.
"But it means that once a company signs up, they want to make as many hires as possible through us.", right, but as a hiring manager that doesn't make sense to me... I don't care that we're making hires to justify where we spend our sourcing money. I care that we're making great hires, so I want to cast as wide a net as possible. I'm not going to shut my eyes to other sources because I've paid for triplebyte. But maybe in bigger companies I wouldn't be the one making purchasing decisions? And the folks making those decisions would be pushing HMs to use Triplebyte?
It is interesting, how the incentives align and don't align sometimes. Hard to say if I'm even in the target market (raised $15m, hiring a few folks this year in the "hardest" category, #1 problem is qualified folks at top-o-funnel).
Maybe I also have a personal penchant for seeing companies who bet on themselves and clearly align incentives :shrug:
I wanted to use triplebyte - but I only want to make 3 hires per year.
-You vet engineers with however you/or individuals engineers see fit (test/ past work/ opensource / interviews/ etc), they highlight what they want + maybe what your data says would help them stand out for the "type" of dev they are.
- force companies to pay you $xk - $1xk to hire them right away if they "match" with someone they select, but with the ability to get a refund if they fire them within 3 months (but like 60 business days to get a refund, and allow them to keep picking but reset the window if they match with someone else again).
- keep in touch with those employees during that window
That way, companies will eventually get their money if they make a bad hire and still be willing to use you again, you can collect more data on the larger part of the employment life cycle and basically get 60 day interest free loans at worst.
We have a big problem in the culture of employing developers that is closely related to how the startup market operates. The market is willing to shovel money at a problem rather than spend time, therefore the market will pay someone with specific experience in their problem far more than pay someone less and have more time to learn the domain. This results in siphoning of experienced developers away from new developers, which artificially reduces the number of experienced devs. The market is creating the problem for short term profit, and then turning around, lying and saying "there's not enough devs".
Yeah I don’t think this is true. I teach undergraduate CS and our data shows (which we publish at admissions so is public) we regularly have close to 100% job placement within 6 months, with 90+% of that being CS related. I know many of the students personally and they are almost all landing good software development jobs. I think we have a good program but I hear the same from colleagues at other schools.
“Good software development jobs” of course is a relative term. CS related does not mean Amazon at >$100k, although a few achieve that.
There's good programs out there, but they are in the minority. If you run a good program, that's great. In my experience most places offering compsci are there to take the federal loan money and hand you a piece of paper.
The reason we didn't talk about it as much in the main post is that our approach to the problem of credentials hasn't changed all that much. We still have quizzes, and someone who is having a hard time getting their foot in the door (a) is likely to be willing to put in the effort to take them and (b) will benefit from doing so. That's not a complete solution to the problem (it helps people with skills get hired, but doesn't in itself help people get skills that you only get on the job), but it's better than the status quo.
We haven't forgotten about people struggling as they look for their first job. (And I certainly never will: to get personal for a sec, my first job search was a miserable multi-year affair that nearly killed me.) We've got some experimental irons in the fire here (e.g. a program that basically highlights people who haven't gotten as much attention as they 'should' based on their skills) that we're not quite ready to talk about in great detail. If they work, you'll hear more about them soon. If they don't, we'll try something else.
This was me when I tried Triplebyte. I wish it had worked out. I don't really blame TB for the failure, but I think there was a lot of optimism on the part of TB that wasn't shared by the companies I interviewed with. TB was just another placement option for every one of them, and I found it challenging to overcome what felt like walking into several onsite interviews completely blind to what was expected of me.
Sorry, but I call major bullshit on this. Sure, if you have a D average at a low tier school, perhaps. But if you are a competent coder who got good grades in school, I see many companies ready to hire new grads left and right (obviously folks who graduated during the height of the pandemic had different issues to deal with). I don't know if we're quite at the "anyone who can fog a mirror can get a job" stage like we were in the original dot com boom, but we're not that far away.
Companies advertise they're hiring. Like they advertise that they're profitable, make the best products, and are better than their competition. That doesn't mean it's true. Many positions are "nice to have", so they will stay there until they get someone with far more experience on paper than they actually need - someone willing to get paid less of course.
People lie, especially companies. That's the nature of the market. Even if we had 5x the devs than employers, employers would still say otherwise. This is because more competition between applicants means you can pay people less.
Infact even during the dotcom boom my sister actually got a degree in "web development", and nobody wanted anything to do with her. I don't know who comes out with these lies about "high demand" employment but so far all I've seen is confirmation bias by those already employed - ignoring all the rest of the people who can't get anywhere, leave it behind, and go work at something else.
https://lh6.googleusercontent.com/5n4R_Osks_fVsVmedxJKsp9zPu...
That's empirical evidence of a tight labor market that's difficult to fake. Why else would salaries rise so quickly from such a high base? Even if employers lie (they do), the salary numbers from BLS are separate from that.
Of course, there are always people who slip through the cracks unfortunately. But avoiding anecdotes and looking purely at the numbers, it's difficult to find a labor market in the USA that's tighter than the one for software engineers.
This can be explain by FAANG paying more, or people being more risk averse of having bad devs (ie hiring less, but paying more for more skilled).
* Reluctance to move / restrictions on where they want to move (only SF, Seattle, NYC)
* Extreme focus on product development (and not applying for the operations jobs that keep many large orgs running).
* Focus on working for a high salary at a big tech company (and not applying to jobs at companies that aren't seen as big tech... say... this job - https://csx.taleo.net/careersection/basic_faceted_search/job... )
I believe that there is extreme saturation in certain desirable sectors of the field and "where is everyone?" in other areas.
I've moved across country twice and from one end of a state to another twice... its a pain, but I like new places.
One of the companies that I worked for (smallish city in rural midwest). There were a few tech jobs there (and a reasonable sized tech company)... several of my co-workers had never been further than about 100-200 miles away from where they were born (and then, only as vacation).
Yea... the reluctance to move is a problem.
I attribute part of this to my (gen X) and the "this is how things are" that I got from my parents (boomers).
My father's family had a few moves through the years... not too many, but a few. My mother's family moved several times (and once even to another territory) as part of my grandfather's career. Once my parents were married, they moved twice before settling down.
For me, when I graduated college, it was almost expected that I move somewhere else for a time. And I did. My brother too... and my sister. My brother (gen X) has moved five times since graduating college and my sister (gen X / millennial cusp) has also had five major moves since graduating college.
I believe there's a generational shift in that expectation of moving out - possibly from the financial insecurities that the dot com crash, '09 recession, and the '20 recession have instilled in the millennials and gen Z as they look for jobs... though that's just a guess.
> rather than looking at the three cities that have the most jobs.
There are a lot of cities/metro areas that have a lot of jobs that people tend to pass by. https://www.bls.gov/oes/current/oes151251.htm - note that there are other classifications that are applicable, but I can't get a page that has a set of them together... so just using this one as an example.
If you look at the raw employment figures, Dallas, Chicago, DC, and LA are way up there too.
If you expand this to the location quotient (based on how many jobs there are compared too other jobs) you'll find many other cities have quite a few jobs too. Huntsville, Durham, and Madison are also up there.
If you allow everyone in, regardless of if they are qualified to be a senior level engineer, will this not just be another job hunt platform where (when I look for another job) I'd still have to do another technical interview at the company itself?
How will companies know now who has been vetted as qualified and who is just lying on their resume? Maybe it's shitty to think this way but there is already a ton of sites out there that allow me to send my resume (or allow a company to view mine) before starting a traditional hiring process that Triplebyte prevented, allowing companies to connect with qualified high level engineers and know the person would fit their skill needs. The way it was companies mainly were just testing for cultural fits, etc.
To be clear, we still have quizzes and the ability to show your performance on them to companies. The fact that they’re not completely mandatory to use Triplebyte doesn’t mean that they’re not important or that we won’t build further features around them.
Right now, companies can offer an expedited process as part of their reach-out. We’re working on ways to productize this a little bit better. We didn’t announce the feature in this post because it isn’t done yet, but one thing we’re looking at is allowing companies to set individual score thresholds for various types of expedited process (e.g. “if you scored a 3 on algos and a 4 on back-end, we’ll definitely get on the phone with you” or “if you have a 5 on python we’ll skip tech screens for you”). We certainly recognize that FastTrack was valuable to a subset of engineers, and we want to recreate the value it offered in a way that is a little bit more sensitive to the specific company and specific engineer in question. We imagine, for example, that less-prestigious companies (who are more concerned about attracting applicants) will probably set lower thresholds than the Apples of the world. Under the old system, we had to set a single threshold, which would necessarily either be too low for prestige companies or too high for everybody else.
And just to lean a little bit back into the pitch we’re trying to make here: if you’re concerned about interviewing process, wouldn’t it be nice to be able to search companies by how their interview process works? That’s not an axis on which companies meaningfully compete right now, but with the right incentives, companies will cut a lot of the annoying hassle that they currently have no reason to get rid of.
If a company asks me if I have questions, and I ask them, and they ignore me, I'm done.
I wasn't looking for a job, but thought the test results would be interesting. Who takes a test and doesn't get results?
I tried TripleByte and wanted to like it, but mostly it didn’t seem like anything special. The whole concept of getting pre-screened, getting multiple offers, and jumping straight to final interviews just didn’t pan out. The last company I interviewed with basically treated my application like any other pipeline. I had a standard recruiter call, hiring manager call, and several technical interviews before getting an offer. And the experience with other companies was similar. From the outside looking in, it appeared that there was no special track for TripleByte applicants. In the end it felt like the TripleByte process simply made the interview process longer, not shorter. Eventually I found it faster & more effective to pursue companies on my own.
TL;DR — companies I met through TripleByte ran me through the standard process I’d experience through any third party recruiter without the personal touch one might get from a good third party recruiter.
TripleByte wants that growth rate more than anything else: therefore, this.
I used them two times. The first one was very early on, where I was given a home assignment and interviewed on it by Aaron himself, if I am not mistaken. The dude was awesome at interviewing, and knew exactly how to probe to get a better understanding of your skills.
That was when they were promising that you can interview with them so you don't have to do technical interviews with the companies. I thought it was an awesome concept and could really reshape hiring in the tech industry.
Second time was a couple of years ago, where the model has already changed a bit. Passed the first round and one of the companies that I could interview for was triplebyte themselves.
What a disappointment! The only difference in the interview process than the rest of the companies was that they gave you a laptop and asked you to do practical coding instead of whiteboard generic algorithm solving.
Some of the interviewers themselves were junior members of the stuff with 0 experience in interviewing.
Scheduled a practice interview and the only slot available was ~6AM my time. Nobody showed up and I wasn't informed that they wouldn't be showing up.
After reaching out about it -- they tried to tell me that my interview scored poorly, and that I would need to retry again in a few months. After some back-and-forth they realized both that it was practice interview and that the person didn't show up. So they rescheduled the practice interview.
When the interview did happen -- this was for ML stuff -- the interviewer was just no great. We spent so much time discussing the differences in terms we used -- mine largely coming from University, there's from I'm assuming their formal education -- that much of the interview was wasted. It literally came down to me deriving what we were not agreeing on for them to understand we were talking about the exact same thing.
I then had to reach out several times for my results, I'm assuming due to the fact that you're allotted one practice interview, and technically I had two(?). When I finally got my results I was again informed that my results were not good enough, and I'd need to wait to reapply. I gently informed them that it was practice interview, and the representative apologized their mistake.
When I reviewed my results... the interviewer didn't rate me too well, largely due to our differences in terminology. They also didn't like my coding style -- even though no one has ever complained to me before -- and rated the coding exercise poorly even though I was able to perform what was asked of me.
I just gave up.
Then several months later, I got an email about being TripleByte certified or whatever.
The whole thing was a really bizarre experience.
is that they say they have companies like Apple and TrueCar posting job listings and hiring but they actually never respond and all you get is offers for companies between 10-200 people in size (aka, startups)
It's a great platform, the indicator letting me know companies aren't going to respond is great.
It just doesn't fix the problem that... the quality of companies on Triplebyte aren't companies most senior/lead/architect/advisory level engineers are willing to settle for. I'm being unnecessarily harsh/biased. I'm sure plenty of people use the platform with great success. I've just come to accept "early/mid-stage startups" job offers from Triplebyte and not much else. Not necessarily "garbage" but... for sure lots of risk.
The "super week" of onsite interviews was interesting, probably increased my overall interviewing skill, but was probably one of the more stressful individual weeks in my life.
People hate overly abstract/CS-focused whiteboard problems (mostly for good reason), isn't practical coding the obvious better alternative?
Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Fundamentally, I think this invalidates Triplebyte's business model. Companies don't really care if someone passes your tests. They still put you through LC type interviews onsite. They simply save a phone screen.
So while in theory I like the idea of this meritocratic minimum bar granting special privileges (I liked it enough to join Triplebyte!), in practice we seemed to be fighting an uphill battle with all but a few candidates each month.
On the other hand, I have wasted a lot of time in conversations with recruiters only to be let down at the end by mismatched expectations. I'm interested to see if this new approach can make a meaningful difference in that problem.
We still have the quiz, and we still show performance on it prominently on your profile (provided you've chosen to share those scores). In fact, we've put a lot of energy into improving the trust companies place in our quizzes over the past year precisely because we think that skills data is important.
> Also, dissing those who have passed the bar as "engineers who like tests" is disrespectful. When they first launched, passing the bar makes you (in their words) a highly qualified engineer, and now passing the bar means nothing except that you only "like tests"...lol
Heh, yeah, that's fair, at least to a point.
Our quizzes are predictive. We know this from a lot of objective data, it's what you'd expect subjectively, and companies do tell us that they trust and value that data. That does not mean that our tests have no bias towards certain personality types. Different people respond differently to testing, and that does have a differential effect, not because our tests are bad but because testing is inherently somewhat artificial.
I actually taught test prep before joining Triplebyte years ago, so I've been this first-hand: it was not uncommon for a student who'd been doing well in practice sessions to crumple under the pressure of the real thing. That doesn't mean the test is bad, it just means that some people fare better on tests than others for reasons other than just their raw ability. When we say "likes tests", that's more what we mean: not that that's the only reason someone does well on a test, but that people who perform well in isolated, pressured environments do better relative to their skills than others.
I think that goes to another ability. If we need a candidate to gracefully handle a 3 AM production outage, we should expect them to handle an interview.
Tell HN: Interviewed with Triplebyte? Your profile is about to become public (1543 points) https://news.ycombinator.com/item?id=23279837
Tell HN: Triplebyte reverses, emails apology (1030 points) https://news.ycombinator.com/item?id=23303037
Because she fucking nailed it with Triplebyte.
I used Triplebyte. I really liked it. The leads were not as great or as numerous as I was hoping, and I felt like I still had to do pseudo-technical interviews anyway. Fucking COVID, man.
I hope this pivot works for them.
> Triplebyte has hundreds of thousands of engineers on our platform, and that means we can flip the script on companies. The collective power of thousands of engineers is enough to change their incentives in a way that individual engineers cannot.
> We can change their incentives directly by rewarding or punishing certain behavior. For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings. When a company’s access to thousands of engineers is on the line, their incentives are very different. The same goes for honesty: a company often has no reason to be honest with any one engineer, but we can disincentivize lying by making their behavior with one engineer affect their access to the next.
So... a union? In digital, online format?
I'm glad that someone there is now realising the advantages of joining forces to negotiate workers' conditions. It was about time high-tech engineers noticed this.
So hiring through TripleByte would be conducted in some fixed-length time intervals (e.g. 1 month). Companies specify skills they're hiring for, interested engineers interview for the skills that TripleByte can screen for and supply a minimum salary (invisible to companies), and companies hiring that month bid starting at the min salary for the engineer with all other hiring companies.
If the market determines the hiring timeline, recruiters can't put you on hold if they're interested-but-not-sure, and can't use exploding offer tactics. It also pushes companies to be more honest about salary ranges and required skills. TripleByte would be responsible for providing feedback rather than the companies.
I found an awesome startup to work at. And now, years later, I work at a FAANG company, working on tons of interesting problems. Thank you triplebyte, and good luck with the pivot!
I love this, but I'm curious about the incentives here. Triplebyte only makes money from companies, not engineers. In the long run, can engineer-centric intentions override a business model based around companies and recruiters staying happy? I hope so!
https://news.ycombinator.com/item?id=27501675
Our bet is that LinkedIn is going to fragment. They are just not creating a hiring process that most engineers like. People tend to either get ghosted, or overwhelmed with low-relevance inbound (almost no one gets the "right" amount of attention). Companies need to go where the best engineers are, not the other way around. So I think our long-term incentive is to fix these problems.
In any case, I'm committed to giving this a try. There is danger that we get pulled toward building for companies. I want top guard against this by being public about what we're doing, and "showing our work" as we go.
> There is danger that we get pulled toward building for companies.
A common phrase thrown around with free services is "if you're not paying for it, you are the product".
> For example, companies aren’t normally incentivized to provide salary and culture data. But we can force their hand by promoting transparent companies in our search rankings.
Maybe I didn't read the article carefully enough, but are you planning to continue charging companies $15k - $30k for the ability to access candidates on your platform?
If so, companies are still your customers. And if you're building and optimizing your product for people who aren't your customers, your real customers (companies) may not be happy with you which will hurt retention, etc.
Maybe I skimmed the article too quickly, but it seems strange to charge companies $20k+ to post a job on your platform, and then actively do things that "force their hand". It might be a net benefit to the engineers on the platform, but I wonder how it will work from a business model perspective since you're potentially creating adversarial relationships with your "real" customers (companies paying you to access your candidates).
Edit: But maybe it's by design, if you actively remove employers who aren't abiding by your philosophy. Although again, that means turning away customers, which means turning away revenue, which in my mind raises questions about the overall business model. A lot of conflicting interests.
That does not fully express my motivations. I am an engineer and find the idea of making the process better for engineers more exciting than making it better for companies. But it explains how I think the incentives work.
Triplebyte has always seemed scummy and this obvious--and unacknowledged--contradiction makes me trust them even less.
We do also have differentiating features on the company side. It's not like we're abandoning the idea of building features for companies or anything.
The job search, like other matching problems, is often not zero sum. So building (say) a better search interface for companies benefits engineers as well, because both sides of the market have an interest in a good match. What we're saying here is that in the cases where things -are- zero sum, the power - and therefore the incentives - lie with engineers and not with employers.
maybe (and i know that this is an outré thing to say these days) that was the point.
To use a rough biological analogy, I think it's generally a lot easier to have a big, rapidly growing company if you take an r-selection rather than a K-selection approach.
At least good companies do this anyway.
Also, executive search recruiters kind-of do what you're describing. Exec leadership is a small-ish world, and the recruiting cycle for, say, a CTO can take months. However, there's a lot of commission on the other end of that. So recruiters basically act as brokers.
I much prefer the imperfect systems that at least pretend to be fair and objective-ish.
On the other hand, when I applied to TripleByte, their feedback to me was "we think your skills are fine, but we want somebody who can perform well in an interview". Which is exactly the opposite of the value you're attributing to them here.
Quoting them, for reference:
> We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured regular expression matcher. It was especially impressive how much you dug into the academics behind regular languages.
> However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview.
- In borderline cases, we didn't want to risk our credibility and companies didn't want to skip tech screens. So competent-but-not-amazing engineers got shut out.
- Our quiz is not perfect, so engineers who didn't fit what we were quizzing got shut out.
- Not everyone needs us to vouch for them. Companies (reasonably) trust that someone with a degree from a top school and years of prestigious experiences can probably do fizzbuzz, and demanding that they prove it was a barrier to them using Triplebyte.
We think it's better to have more granular ability for companies and engineers to decide what mattes to them. A prestigious company can say "we'll only talk to people who took the quiz and got a top score" (and that score is us saying "this person deserves a job"). One that desperately needs the headcount can talk to the borderline cases or decide they don't want to put the extra barrier of a quiz in the way.
Like most markets, we think the hiring market on our platform works best when it's able to respond to local conditions.
If you're not willing to refer people to companies based on their score, how can you interpret the score as you saying "this person deserves a job"?
How does the new ability to make that recommendation address the original problem?
Edit: just saw the comment that TB is keeping the quiz. Would consider using the platform in the future as long as that pre-qualification track is available and that this new direction is just a superset on the existing TB features.
Do not believe anyone who tells you they can force employers to change their methodology when it comes to salary and compensation revelation. Recruiting agencies are so far outside that wheelhouse that it's damned near scandalous to even pretend the statement has any validity.
Also, I think it's important to differentiate staff recruiters from outside recruiters.
Staff recruiters are HR people. They're on the job to shove square pegs into square holes.
Outside recruiters are salespeople. They're on the job to explain why a round peg can reasonably fit in a square hole.
This is your career. So act like it.
If 10 different people were each going to give you a hundred dollars if you described yourself to them in a way that would obviously ingratiate yourself to them (person with cat photo, person with dog photo, person with photo of their kids, etc) most would immediately come up with reasons to make themselves1 appeal to their tastes.
But people keep on submitting the same dumb resume over and over again and expect a different result. Accentuate the things in your resume that matter to the company you're trying to get a gig at. Yes, for each of them. If you would do it for a grand, you can do it for a career.
1) Look at relative data, not raw ratings. Written reviews or raw numerical scores are what show the nightclub effect the most. What we care about is a ranking of companies (showing the better companies first in search results). This may not be as impacted by the problem. If sour grapes from people who failed interviews overwhelm other signals, we can only look at the scores from people who pass (or normalize the two groups separately).
2) Ask very specific questions (and maybe provide dummy options to attract sour grapes). I suspect that questions like "did this company say they could meet at $250 salary and then offer you less?" will get more accurate answers than "does this company have a toxic culture?". (We do actually want to get culture data, but I want to be really specific about the culture questions we are asking).
3) Use objective facts, not subject opinions. For example, we can tell when a company ghosts a candidate (because they reply through a proxy email that we control). So we're listing the 'ghost rate' for each company.
I'm pretty sure many salty people will just lie anyway, to punish the company. Source: I've been on the internet a long time.
I think this model of 'punishing' companies would've worked when you had a high bar for being on the platform in the first place, but if anyone can get on there, you'll have a lot of low quality candidates who are very mad that nobody will take them, and they'll overwhelm the reviews.
Remember that there's a survivorship bias at work: the lower quality people will take longer to get hired, which means they do a lot more interviews, which means a lot more reviews on your platform. Essentially, it's a reverse meritocracy when it comes to evaluating companies.
Anyway, a ghost rate would be greatly appreciated.
Maybe this would be useful for folks just starting their career, but on the hiring side of this, in the last year I haven’t made an offer to a candidate who didn’t already have at least one other offer on the table.
Overall I think there are a lot of misconceptions about how people are hired from the recruiter side — which is why most of these platforms see the recruiter as the target customer, not engineers. I get that y’all are trying to change that, but without insight into the hiring practices at any given company (e.g. my company gives a lot of latitude for salary negotiations but every hiring manager negotiated differently) it’s going to be hard to pull a signal from the noise.
That being said, the experienced engineers who can get that kind of attention have other desiderata in their job search beyond just "get an offer". "Get -an- offer" is a goal more typical of a new graduate or someone trying to break into the industry than it is a goal of experienced engineers.
To people who are sure they can get a job, there's a big difference between a 150k offer at a company that has none of the workplace traits they want and a 200k offer at a company that they'd genuinely love to work for. And right now, they can't necessarily tell the difference between those jobs before going through a bunch of application steps. The example we mention in the post - of a highly credentialed and very senior engineer who kept having his time wasted by recruiters who refused to admit that they couldn't offer what he was worth - wasn't cherry-picked. It came up organically while we were doing user research, because it's a really common problem, and was echoed further by a number of other senior engineers we spoke to.
(Of course, we care about the folks just starting their career, too, so I don't want to totally ignore the importance of solving problems for them. But I think we are solving an important problem for senior devs here, too!)
We look at the job posting as a brand marketing — all we’re really trying to do is get you to apply and fill the funnel. The whole hiring process is designed to be a calibration on role, cultural fit, compensation, etc.
Seems almost like the bait and switch that the OP company is railing against in the blog post.
I'm curious to see how you guys have managed it without disgruntled employees quitting within 18 months.
I work in consulting so high turnover is just expected. We’re actually a lot lower than the industry average. The job isn’t for everyone, but there are people who thrive in this environment. For everyone else who isn’t a good fit, we get a year or so worth of billable hours, they get exposed to a lot of different scenarios, and we give folks a lot of runway to find something they do like.
It’s just the nature of the beast though. If I get a candidate who is legitimately at level, I bring them in at level. There’s no need to be shady about it, but job search sites are what they are and that’s the game you play.
Anecdote: Recruiter R from Company A reaches out to me, we have a conversation and she says HM (hiring manager) loves your profile and wants to talk couple of days later and so we schedule a time slot to talk to HM. Next day, R calls and says HM wants to know why I left a previous employer X and I just blurted out the truth which was that X made it very difficult for me to take FMLA when my kid was born and I got pissed and left. Shortly after this I received a rejection message. This is classic discrimination as defined by EEOC.
I replied to R and said this is discrimination and she denied it. I posted my experience on Glassdoor and my post was flagged and removed. Nowadays I am honest but not truthful.
No response after getting the "wow, nice quiz result" email. No further emails. No replies for months to my gentle inquiries. Eventually they did reply, saying they were sorry but they didn't need help.
I have significant doubts that they were hiring tech interviewers in the first place. I think instead they were gathering market data at our expense.
Furthermore, there were detailed stories from users on Glassdoor about their internal policies and attitudes, mostly painting their leader as lacking scruples and leadership ability. Perhaps this was all fake, but if so it was really convincing and thorough.
I'm curious if anyone has had legitimately good experiences with them.
I have started to re-write my resume for each application and that has yielded much better results in terms of quality of the company and number of interviews.
If you are submitting the same resume to multiple companies, you are doing it wrong.
But glad you figured out an approach that worked for you!
It would be a nice touch though.
The hiring manager side of me on the other hand continues to remember that mishires are expensive. I've also very skeptical in that once a month some staffing agency comes along, tells us they have some fancy heuristic for matching good candidates to our jobs, and then unleashes a tsunami of mediocre leads into our recruiting database. They'll then proceed to act upset and surprised when we don't want to intervene sixty percent of them and most of those we do interview wash out. I guess TB can somehow promise that a certain baseline technical competence is present, but even that's of moderate value. If I really need a candidate who is skilled at X, I might be willing to sacrifice some skill in Y because I already have two engineers strong in Y. I never found TB sufficiently granular in how you configure the automatic screens to allow for that. Secondly, it's totally useless in soft skill assessment.
The cynic in me just sees a lot of this TB now trying to force you to expend internal resources to rank the unqualified people or people whose skill sets don't align with what you want (forcing you to respond with data on how you handled the application) because they can't crack the nut of doing it themselves. So they're going to leverage the customer's recruiting teams to code the data for them.
I had some bad luck. My nodejs build broke after a recent update on my machine, which I didn't realize until right before the interview. During a test with a different language, I tried to define a constant with the same name as a built-in function and ran into a vague compiler error (something about missing parentheses, ugh). This language has case-insensitive function names to compound the confusion. I unfortunately looked up how to define a constant in the docs. Their conclusion was that "I was uncomfortable in the language," despite having used it for 10 years. There was some other feedback which I felt was inaccurate, I think I just had a bad day, and obviously didn't convey my knowledge and experience well. I could see why a recruiter would hard pass on me for some of the stumbles, since their main goal is to forward candidates who interview well. It hurt to get rejected.
I mean, you went into a combat scenario, and chose a weapon absolutely notorious for jamming.
If a company has a bad interview experience, a bad culture, or the management is impopular, if the work life balance is bad, or anything... you'll be able to read about that in those websites.
For example, a recruiter contacted me last year regarding an opportunity for a fancy senior role at a large company (Sr Staff engineer)... I checked the Glassdoor reviews for that company and found many testimonies from people who were contacted with the exact same narrative, but at the end of the interview process they were offered vastly lower levels, so clearly that was their intention all along. Because of that, I assessed that interviewing there was going to be a waste of my time and I ghosted the recruiter. Developers win.
Of course, you need to take some reviews with a grain of salt, but if everyone is saying the same thing, you will at least have a sample of what you are getting into.
Having said this, I am not sure Triplebyte will ever be able to effectively solve or mitigate the problems Ammon is describing. It is possible that he needs to continue talking to developers, and I am not trying to say this in a rude way. There are also many more problems that need help fixing that Ammon did not describe.
Having a conversation with a developer about different topics can give you some insights on the breadth and depth of the knowledge of a developer, and many interviews omit this. I think interviews should be "adaptive" tests, that get as progressively more difficult until you reach your limit.
I recommended TripleByte to several friends whom I considered reasonably skilled and currently undervalued. None of them passed the test, however, so they got little value out of service (except for the interview feedback, which at the time was unusually excellent). And I ultimately needed TripleByte less than they did, since someone who could pass the old TripleByte test also has a high chance of eventually passing on-sites at Google, Facebook, etc.
So I understand the pivot, although it will certainly be harder to differentiate yourselves in this new space. Good luck.
Triplebyte has existed for, what, 4 years? That actually doesn't sound like very many.
> Triplebyte has hundreds of thousands of engineers on our platform
Hmm.
For unemployed candidates, we should do away with interviews altogether and do paid 2-week trials instead. After the first 2-3 PRs it's pretty clear if someone has the skills you need and to what degree. It would also help the candidate decide if they like the company.
Employed candidates are harder to evaluate and I don't think there's a single formula that will work for every company.
My takeaway from my experience finding a SW engr job was that actual technical capability was really only a second- or third-tier consideration in companies' hiring decisions. (This is often misaligned with the companies' actual business needs, but that's not what materially drives hiring decisions either.) So, although TB nailed TFOO the competence assessment & pre-screening problem, Amdahl's Law still applied to the whole enchilada.
What TB should do is monetize their experience placing candidates over the last 5+ years and just sell training & maybe some standardized SaaS products to HR departments -- especially at startups! -- that regularly need to hire SWEs/SDEs (yes I'm a FAANG-er now). Because the real dysfunction in tech hiring originates & thrives in those departments, and...something about robbing banks because that's where the money is...
That capable people failed to make one solution work, doesn't mean no solution can work. Now they're going to try another.
That's a lot less than I expected.
What am I looking for?
1: I want my skills evaluated in a standard way by some neutral licensing board, just like engineers who build dams or surgeons.
2: I want job interviews based on mutual interest, not proving competency.
3: I want my colleagues to be certified, just like me.
4: I want legal standards in place that give me leverage against employers who are cutting corners, or managers who "know enough to be dangerous."
Example: My wife, a doctor, had to go through an intense process to be certified. The vast majority of medical school graduates pass certification. She only applied for a handful of jobs. The interview was based on mutual interest because the certification process proved she was competent. All her colleagues are competent, and her hospital is well run. There are legal guidelines that make it hard for her hospital to cut corners or make her do something unethical.
For most developers, the worst they can do is bankrupt the company they work for, and even that usually requires multiple compounding screwups and a bad luck roll.
I personally think the gap between our standards for doctoring and our standards for software development makes a lot of sense.
I do think people building life-sustaining software should be held to a higher standard, but that's pretty rare.
Also, are we sure there is a shortage of engineers, or simply a shortage of engineers that can pass the hiring interviews and qualifications?
I think a lot of companies don't always mind waiting to find someone who passes there test, be it a newer grad, or an existing engineers that has now prepped up better.
I still don't understand exactly what benefits it brings to me as a programmer though. I'm about to start preparing for interviews with 25+ years under my belt, after not working for 2 years and I've accepted the double-edges nature of these interviews. I know I need to LeetCode for 3 months to be competitive and memorize as many of the answers as I can. I know that some companies will ghost but others will be forthcoming. I know that in big companies, different teams have different recruiting processes, so it's hard to put an entire company in a single basket.
I have friends at all FAANG, and they are very open with their salaries and TC, so I don't know what Triplebyte would bring to the table besides friction, so be perfectly honest.
In addition, as my username suggests, I really like the idea of a representative for Engineers during the job search process, but I also want to make sure I'm the customer of this representative. If I'm paying TB then I think the incentives are aligned. If they're just a job board then their new stance is opposed to the interests of their true customers (hiring companies) and that won't work out too well for them.
My firend cracked interview and joined one of this and for others like me have no traction (6 months in) on the resume at all after using hired/linkedin/direct careers/cold emails. Doesn't matter you have github with real projects/portfolio with good responsive design .......
If this process is broken and we also don't know what are we doing wrong then how all them are going to find all these talent they need??
The interview itself was nice and ok, the feedback was really appreciated (after all this more or less reflects what high paying programming recruiters are looking for). But I felt like there was no place for people with my background.
I was rejected when I did the quiz at university, trying to get a job.
What made me a bit angry is how happily they rejected me with an automated email, after I put effort and time into doing their quiz... I didn't feel like recommending them, or even visiting the site ever again.
Glad to see they're finally aware of how this approach doesn't benefit them as much as it could of if people were still invited into the system.
What Triplebyte is trying to do is give more information back to job seekers. But it will be limited to the companies they actually work with (and punishing your own clients sounds like a peculiar business model...)
Why not build a Glassdoor for hiring, that would cover the entire industry? Anyone can go and describe the hiring process they went through, if they heard back or not, etc.
Does such a thing already exist? If not, why not?
I felt like Triplebyte worked hard to create a good experience for me, but that overall the companies I interviewed with just viewed TB as another placement company. Other than getting to skip the fairly easy phone screen phase, I didn't feel like I got much value from the experience at all, and I had to put a lot of work into it.
Ever thought of building a recruiting recommendation software product, instead of just selling people? You have the data.
edit: I want to be more constructive with my feedback. You have four years of data on hundreds of thousands of candidates. I don't know what the legal implications are regarding using that stuff as training data, but.. embeddings. doc2vec. Thank me later.
The quizzes weren't quite a fit for ML, nor was the interviewer or job density there for the domain, but the immediate half-hour call with the promise I'd passed a technical screen was extremely efficient, even if Triplebyte lacked any basic level of market liquidity.
I can also report I haven’t received a single TripleByte email since the issue was last brought up :) Thank you for standing by that.
Copying this from the article. Doesn't this sound like a union? Unions generally lead to mediocrity though.
Does this not mean that even if you apply from TripleByte that you now still need to go through FizzBuzz and all that?
So it sounds like the hiring manager can choose to skip a fizzbuzz call when they know you already passed.
Long story short I bear no ill-will towards the people at Triplebyte but will not attempt to use them again.
It's a tricky situation because you have to balance being accepting of people with non-traditional backgrounds while also making sure they can do the basics. I liked that TripleByte could condense DAYS of interviews into a few hours.
In general, long tech screens are hard on people that can't afford to take off a full day, so it's tricky for people already working in crushing jobs that need a change. The TripleByte quiz worked nicely for this.
On the flip side, if their tech screen is filtering people for reasons other than ability, it's removing possibly good candidates who deserve a shot at something.
The new approach is sounding like it's trying to satisfy the latter, which is good for improving inclusivity, but won't be to everyone's liking.
"Triplebyte, to my mind, did an admirable job of trying to solve the credentialing problem. But their approach was not without shortcomings: 1) not everyone wanted to take their lengthy quiz, even though the quiz was well done, and 2) scaling up an army of interviewers, all of whom had to be trained in exactly the same way, was non-trivial and not cheap. These challenges were surmountable, but the challenge that wasn’t arose from the second issue that all companies had to face: lack of candidate autonomy, which was driven in part by a lack of faith in the credential.
Once you passed Triplebyte’s assessment process, just like at Hired, you had to interact with a talent advocate (Triplebyte labeled them “talent managers,” but again they’re just recruiters). The talent manager would examine your background and short-list you for some companies of their choosing, where you’d then go onsite. You could have some input into which companies you spoke to, but it was limited, and if you didn’t meet the company’s (often somewhat arbitrary) criteria, no matter how well you did on the assessment, its doors were closed to you.
As with Hired, having an army of recruiters AND interviewers working for you makes achieving SaaS margins impossible, and then, you’ve essentially become a tech-enabled recruiting firm (albeit this time one with much better performance data!).
Just like Hired, Triplebyte eventually moved away from the auction marketplace model, fired most of their talent managers and interviewers, and fulfilled their destiny, becoming a glorified LinkedIn Recruiter clone. Recruiters could search for candidates, just like on LinkedIn Recruiter, based on their pedigree, languages known, and so forth. One thing Triplebyte still does differently, however, is to leverage their aforementioned coding quiz to annotate candidate profiles. (Presumably, they are using their historical interview data, from when candidates had to do BOTH, to predict how people will perform in interviews.) The limitation, of course, is that great people will be unlikely to take the quiz in the first place, especially now that it no longer fast-tracks them to an onsite."
https://blog.alinelerner.com/ive-been-an-engineer-and-a-recr...
https://news.ycombinator.com/item?id=23279837
“ In short, there is more demand for engineers than there is supply, and that makes engineers powerful.”
This statement is very telling that TB leaders got tunnel vision, a very SV, startup, software centric tunnel vision. Let me rephrase your statement to what you really meant.
In short, there is more demand for “experienced Software” engineers “in startups “ than there is supply “in Silicon Valley”, and that makes engineers “,with pedigree of FAANG or top schools or well connected,” “very” powerful.
Not that there's not room for another player in the space, but it's just a good example of how to do it well.
So unless there is a simple business model shift to charging the candidates, all of their words are empty. THIS is why they made the mistake of making profiles public, because they were thinking about their customer (THE COMPANIES) and not their product (THE CANDIDATES)
"If you're not paying for it, you're not the customer; you're the product being sold." Tim Oreilly Tweet
Sidebar: Take the Quiz!
Maybe they should rollout the update before announcing it?
The biggest problem with engineering job searches is that they're a huge time sink. I'm a senior engineer with Google on my resume, so I don't have much trouble getting in the door or passing interviews, but the process is such a slog. If I were to take PTO for a week and cram my schedule full of job search, I could probably get 3-4 interview processes done per week. That's not a lot, considering that it's good to line up offers and that many people have limited PTO. Even if your PTO is "unlimited", it looks suspicious to be taking random days off for on-sites.
What this means is that I try to keep doing my job and schedule phone screens and recruiter calls in the gaps when I know I won't get meetings. I try not to have to block time on my work calendar for these in case someone wonders "why does Troy suddenly have so many random meetings all of the sudden". The scheduling process with companies is a pain, they ask for 2-3 time windows for the phone screen, then often take a couple days to pick one. So I have huge time blocks reserved for them that I can't give to other companies I am interviewing with. It's stressful to try to slot all this in. Add to that the fact that recruiters want to do everything over the phone (which is very unnatural for me), so I have to also schedule little interruptions for 15 minute "chats" to "check in" and prepare me for the next round. I appreciate these things but at the same time, I'd appreciate fewer of them.
When it comes to the actual interviews, some companies seem to be fine with one phone screen, some need 2-3. Sometimes they are an hour, sometimes 45 minutes. Some companies see that I work at Google and deduce that I might be competent enough to skip the phone screens entirely. I appreciate that.
It's the same with on-sites. Some are a half day, and some are more of a full day. It's not a big difference in practice because I couldn't really take a half day off from 11-3. Many companies have a "no interview day" which is great for their employees but hard to work around, particularly if it falls on "non-suspicious" PTO days like Monday or Friday. Sometimes I wish we could just do several phone screens instead to make scheduling easier.
I'm not sure how to solve this, but it's the biggest pain point by far and if someone could solve it I would absolutely use their platform. I want fewer phone calls, fewer hours of interviewing, and a clear understanding of the time commitment ahead of time.
I also wish companies would be up front about WLB so I don't waste time interviewing, only to learn they work "maybe 50-55 hours a week". That's not a "little more" than 40 guys, that's more than 6 full work days every week. You can't just ask at the outset because some people perceive that as a red flag.
I can't believe this. I must be a garbage engineer then, because I found the US job market for software engineering roles to be absolutely soul crushingly difficult. I am not the only one, my friends did too.
I have given up several times and then restarted job search. I have been grinding leetcode but companies want nothing less than complete perfection.
I actually thought something like triplebyte would be useful, give a test once and have the score be valid for a couple of years, like a GRE for algos and data structures.
I really don't get it.
The "solution" to the big problem might be a 10 page PDF (for reasonable fee or free) to engineers on how to play the game to their own advantage. Of course no one really profits big from this enough to keep a startups lights on. But if all developers played the game well it helps everyone, even the few that don't play it (like a vaccine).
I wonder what 'fad-diet' solution this company I've never heard of is pitching for either (A) getting me an engineer or (B) getting me a job as an engineer. Oh, let me guess, you're going to find those magical engineers who come from non-traditional backgrounds or magically making it possible to both "identify the top 5% of Ruby on Rails developers with no fewer than 2 years of React Devops Cloud Experience"[0].
Going from that, if you RTA -- I was pretty horrified to hear what they thought would be effective in this space. I realize you'll have to take my word for it that there were literal sentences in there spoken as "good ideas" that would have literally had me thinking "oh, so this company's some form of scam" had I happened upon them before they chose to reverse these ideas. I gotta say, public profiles, entrance exams in an attempt to "hack hiring"[2] don't know what this company is. I have to take everyone's word for it that they're somehow big in this space as I haven't looked for a job, nor been responsible for finding viable job candidates[1], so that's probably me.
I was kind-of torn about his apology. It was really un?-professional (I mean that as a compliment but I can't find a way to say 'not corporate-fake' either in 'sorry-not-sorry', 'deflection' or with what appeared to be much concern for what the lawyers have to say on the subject. I don't know that the lawyers would have to say much on the subject but I'm told that this is often the reason why companies manage always manage to communicate "Sorry,... (assholes...)" every time they apologize. There's always that miserable thing, they're going along "We apologize to our customers ..." it's looking good "... for the inconvenience this outage has caused." Seems simple enough, except you've taken no responsibility -- the outage was to your software, or your hardware, running on wires that you pay for which you are supposed to have redundancy for, and all of this stuff is outside of your core competency but there are many companies who can help if you're willing to spend. And I'm sure when the issue happened, there was a key choice that had you spending a large amount of money and me getting a flight home; which you also chose to inconvenience me over.
But Triplebyte wasn't Southwest. I think the author went a little over the top on the apology, almost crossing into the "trying too hard", except that the things he was apologizing for -- basically, being blind to some pretty obvious problems -- were kind of spot on. Please don't think I'm trying to throw some sort of slant compliment; I realize I sort-of sounded like a dick.
There's no malice in that statement; I've been more blind to things I should have know that were right in front of me than a silly business model decision made too quickly. I am certain I have not apologized this clearly/bluntly.
And the thing of it is, if it hadn't been done so well, I probably would have read the first two paragraphs, closed the tab and made a mental note to never touch them -- basically, having them go from "I don't know who they are" to "probably a scam". It's the risk a company takes when their apology gets attention beyond "people who (currently) care". I am not currently hunting, but I'm all around it on the "need to hire someone" side and I have no guarantee I'll keep my current job, forever.
Thinking about it -- I've now read a very long post and written a very long comment about a company that I had never heard of and upon hearing about them, couldn't care less about their services (the space, yes, deeply). They'll probably be the first address I put into the bar if my needs change in the next year or so because I can't name a single other place I'd actually look. This guy might be an evil genius; if I suddenly have a strong desire to change employment in the next three months, I call witchcraft and there'll be pitch forks and hay, or...something. :)
[0] I must have been in a really bad mood because HN has never been a place that promotes that crap to the level of this post, so that should have been my signal that there was something of value here, and I'm glad I stuck it out.
[1] Mostly only interviewing; we have a pretty spectacular individual who has extensive knowledge about hiring for the rather broadly scoped kinds of work that just my tiny team in the 100-or-so company that employs me. I have a process that, though I am always tweaking, the major points are about as perfect a process. I've commented like this on that subject many times in the past if you want to be super bored. :)
[2] I'm confident in my abilities as a software engineer; push comes to shove, there's many, many ways to make enough money to live reasonably in most parts of the US. I've never had problems making money writing software, either directly for an employer or on contract, independently. Neither my wife nor myself have completed a 4-year degree (though I was one semester away; long story), she did a job that does not require a degree -- became a Realtor -- before we met, six-figures in 2003? I think. The most Senior guys on infrastructure pulled that where I worked (they paid a little under average for the role/area but like 1-2%).
i'm not sure about the viability of their "search criteria" though - things like test coverage, release cadence etc are team specific, so it's hard to define it at a company-level, other than maybe small startups
my advice for triplebyte is to partner with leetcode and establish an industry-wide coding certificate, kind of like those SAT or GRE tests that you pass once and apply everywhere.
"Jesus this guy writes and writes and says nothing. Wait, there was a link at the top to some sort of TL; DR, let me check on that to find out what the announcement is"
> What are we pivoting towards? To explain, let’s talk about economics for a second.
Nope, I give up.
Take a look at the question they just asked me when I was doing their General Coding Assessment:
The only thing that bugs me about that question is that "the output of the following function" is confusingly worded. It's the sort of question that a competent candidate might get nervous about and start to overthink: wait, that isn't a function, it's a code snippet. The function here is foo – so maybe they mean "what's the output of foo"? But then what does "the output of a function" mean? I suppose they mean return value? Is this a trick question to see if I remember what push returns? Damn it, does it return the entire array or just the pushed element?
That's what my mind usually does with questions like that and I'm far from the only one. Since the goal of the test is to screen for basic competence, it ought not to filter out people who could answer the question perfectly fine if it were being asked clearly, but who also perceive corner cases and ambiguities. Such a skill should make you more likely to pass such a test, not less. Therefore the question ought to say something like "What does the following code snippet write to the console?"
The timer is what is bad about this.
I do not care about the question.
As you say, any competent programmer could solve this in a few seconds, if they are proficient, or minutes if they are not.
In my opinion, they should not put a count down because they are interrupting the programmer’s flow. If they really want to measure time, they do it without interrupting the programmer, hide the timer, let them take as much time as they want/need to solve these problems. Then, at the end of the quiz, show them how long it took them to give a proper solution, and take in consideration that time to score them.
If Alice solved 10 problems in 10 minutes, and Bob solved the same problems in 15 minutes, maybe there is something there that is worth highlighting in their profiles. Maybe Alice is able to analyze this type of questions much faster than Bob, and if that matters to the recruiters or potential employers, then allow the candidate to use that in their favor.
However, if Bob actually knew the answer, but they ran out of time to select an option, that seems unfair.
You've found exactly what they are looking for.
Also, it's pretty bad manners to leak test questions imo. You probably agreed not to in the TOS that you accepted.
That's a tricky question, and if you're not good enough to understand why they're asking that then you fail!
There may be some edge cases I'm not covering here, but JavaScript passes primitives by value (strings, numbers, etc) and non-primitives (objects, arrays, etc) by passing a copy of the reference.
What this means is if you reassign the reference inside of the method, it will only affect that scope, because the reference itself is a copy. If you modify the properties of the non-primitive, it will be modified, because the copy of the reference points to the same non-primitive.
Since primitives are passed by value then any modifications are not reflected outside of the method.
To the other poster below, in JavaScript, const with an array (object, etc) simply prevents reassignment of the variable, the array can still be modified.
a and b are shadowed in the foo function so the function only acts on the shadowed (and copied) variables. Once you return from foo, the const a and b are unchanged.
Not a great question as a pass or fail test imho because if you use shadowed names often enough to be able to parse this code in your head, you’re writing bad code. There is a reason why shadowing names is a bad practice: it’s hard to figure out what the end result is and account for side effects! But as a discussion on all the points above then it would teach the recruiter something about what the candidate knows rather than using this as a trivia question.
At least I assume that's how const works; I've not used const in JS.