Behind the Scenes: Hiring a Threat Researcher
Peek into Pulsedive's Threat Researcher hiring process from start to finish. We share tips and lessons learned for both job seekers and startups.
A peek behind the curtains of Pulsedive's "building the plane in the air" hiring process, with examples and details of each step along the way.
If I post, will they come?
Before I hit post on Pulsedive’s Threat Researcher consultant listing, I was cautiously optimistic about the response we would get. Little did I know we’d be blown away by the caliber, passion, and interest by the candidates who applied.
A few of you may recall how LinkedIn paused our free listing in a matter of hours because we hit a previously undisclosed limit before paid promotion became mandatory. Funny thing - when I was about to hit post, LinkedIn’s infinite algorithmic wisdom estimated only “5 applicants” a month unless we upgraded to the paid tier to get 50 a month, not 50 an hour. Luckily, we had many respondents who also came through our blog, applying directly to talent@pulsedive.com. We ended up organically accepting ~205 applicants before we closed the post off. (See our full talent funnel, with numbers, below).
Sharing how the sauce got made
Since, I’ve had colleagues reach out with friendly curiosity around the process and for details. Founder peers looking to hire but not sure what the market is like. Researcher friends who aren’t looking just yet for an extra gig but wonder how they’d fit. We’re a startup, building out processes, many times for the very first or second time… and making benchmarks to assess whether we were successful and how to improve the next go around. So, I’m sharing some of the unfiltered “behind the scenes” experience in hopes to:
- Demystify the startup hiring side for candidates (in general and for Pulsedive)
- Help other early startup teams who are looking for real examples of how it can be done extremely lean/bootstrapped (this is the life we choose)
- Transparently share some numbers and processes for feedback and improved practices all around
The effort from planning to signing was 10% strategy, 90% sweaty, sweaty execution (as is the case with most startup things). If you’ve been in a hiring process before with a smaller team, none of this will come as a surprise. You’ve been warned.
Below, you'll find an overview of our schedule, key milestones, and funnel.
Timeline to Hire
We kicked off this initiative in earnest after our 2022 year-end strategic business review. Fortunately, we kept mostly to our schedule, with a few extra weeks added on the end during final reviews. Juggling clients, events, product development, and *stuff happening*, managing priorities and keeping your eye on the hiring prize is easier planned than in practice.
1. Establishing the Why (Rationale)
We start each adventure with short answers about why we’re creating the role, its strategic importance to Pulsedive, and opportunities available down the line. We also cover logistical details, including:
- Prep and resources required to support the role
- Reporting relationships and accountability
- Proposed timeline to hire
Early alignment helps avoid misunderstandings or crossed wires after a bunch of invested effort. Getting good Threat Researcher candidates in the door was one of my top priorities for Q1, so this was a no brainer. Here's a snippet:
One of Pulsedive’s priorities is to provide current, high fidelity data to our user community. While we have automation and new data projects in place and on the pipeline, much of the curation and value of the community database can be enriched through intelligent human processing.
This part-time role will address opportunities in improved coverage for current threats, reports, and content that is TLP:CLEAR but not currently being ingested. This role will act as a power-contributor to Pulsedive’s public database, provide a practitioner voice to our product, and produce actionable knowledge for the greater community.
2. Spelling out Specs
A lot of work goes on behind the scenes before the job post. While the rationale is a good summary, we concurrently developed a core competencies matrix, scope of responsibilities, budgeting plans, keywords, outreach channels, and legal requirements.
For this role specifically, given that we had applicants who had related full-time jobs, we engaged our legal counsel to ensure that we were super forthright on confidentiality, disclosures, and warranties. For this reason as well, demonstrated integrity and professionalism was a top consideration.
Bonus: I gauged interest from the talent market to inform our spec building. It’s fun to socialize and tease an opportunity within trusted networks as a smaller startup, as you can quickly determine whether or not you’re on the right track by seeing who nibbles and the subsequent questions/comments. It’s my 10% dip-your-toe-in-the-water feasibility test before committing fully to a larger project. The very first time we floated this idea was around mid-2022.
3. Publishing the Post
We iterated through 2 drafts until we got to this post:
Grace Chi
Which was followed up shortly with this Q&A:
Grace Chi
We posted the opening organically on our blog, social accounts, LinkedIn jobs, and also within some private groups. Going beyond what the role was, we wanted to drive into the heart of why it would be exciting to the right candidate. What got us particularly excited was the mix of those who applied - and how many exceeded expectations. We were pleased to find both many long-time users who wanted to work with us and those who’ve never heard of Pulsedive before who were intrigued.
Our LinkedIn post had a click-through rate of 10% and engagement rate of 13%. My personal post about LinkedIn pausing the job post ended up hitting 30K+ impressions… which, funnily enough, amplified the job further unintentionally. Sometimes you can make your own grass greener.
4. Creating Consistent Scripts
We developed the full email and written scripts to templatize the whole process, which included some of the following steps:
- Warm Outreach Email and Message
- Application Received Email
- Application Rejection Email
- Application Rejection (But Future Interest for Other Roles) Email
- Screening Email
- Screening Rejection Email
- Interview Invite Email
- Interview Reminder Email
- Interview Questions (mapped to competencies matrix and including potential follow-ups if missed)
- Final Assessment Email
- Offer Email
5. Streamlining the Screening Process
All candidates were screened with a 0-5 initial rating on base requirements. We looked at resumes, LinkedIn profiles, cover letters, and any provided links (GitHub, blog, Twitter, YouTube) to validate if candidates met a minimum of 3 years in cybersecurity, 2 years in threat intelligence, etc.
Candidates rated a 4 or 5 after initial review were sent screeners, as well as half of the 3's that demonstrated some interesting standout aspect in their application. We also asked about expected compensation ranges upfront to avoid mismatched expectations and wasted efforts on both sides.
Our email screener confirmed their base experience, if we could meet their expectations for compensation, and dug in deeper by asking about the sources of CTI news/content the candidate used and a portfolio of work.
A few lessons learned around Portfolio Review:
- Coverage was very patchy because of candidates who couldn’t share materials or had no TLP:CLEAR portfolio, understandably. This still ended up helping us screen some out while “starring” others who had great content, but left many known unknowns marked for deeper, more targeted interview questions.
- As a result of these known unknowns, we discussed and agreed to fund a paid assessment that could demonstrate live application for our desired Threat Research skills.
- While I created an initial formula for portfolio review, the inconsistencies of what we received meant we ended up with a much lighter-weight assessment. Instead of significantly impacting final choices, it was more of a general thumbs up or down on the interview score.
6. Digging Deeper in the Interview
Our interview was pretty standard, covering:
- Candidate Overview (10 min)
- Past Experience (15 min)
- Applied Experience (15 min)
- With Pulsedive (10 min)
Under each section, we made sure to map questions to our assessment matrix, and had concrete follow-ups for ambiguous responses.
As an example, under Past Experience we asked all candidates the following:
“Tell us about a time you compiled and communicated research findings to different audiences. Who were they and how did you approach it?”
Where necessary, we dug in deeper with a follow-up question or two based on the response or other identified unknowns, such as:
“In what forms and formats did you communicate?”
“How did you go about compiling [insert reference data points]?”
“What were the outcomes for [referenced] audience?”
“What feedback did you receive?”
“How might you improve this process if you did it again?”
These questions helped us address knowledge gaps we had before the interview and share specifics around the candidate’s thought process and experience. We wanted to avoid finishing an interview feeling like we couldn’t make an informed assessment.
7. Finalizing the Hire
After the interview, we expected to have 1 or 2 candidates we’d be excited to bring on. Instead, we were having serious discussions around 5 or 6 and managed to create a finalist pool of 3. To fill any remaining questions and get a realistic skills and output assessment, we prepared a paid take-home report assignment.
Below is a snippet from the assignment and terms shared with the finalists. Since the role was based on the interests and skills of our finalists, we wanted to leave the deliverable format flexible; blogs, graphics, code, etc. were all encouraged if the candidate believed it would add value to the report.
All 3 reports came back more than satisfactory - which was a nice validation of our screening and interviewing efforts.
After final internal discussions, negotiations and letting the ink dry, we successfully brought on our Threat Researcher - a key contributor to Pulsedive’s dataset and research output today... including:
Mohammad Amr Khan
Mohammad Amr Khan
Mohammad Amr Khan
Miscellaneous Notes:
- Cyber people, and many others, love remote, work-as-they’d like opportunities (duh). This helped the role get a lot of interest and allowed us to compete against bigger companies with much more appealing perks and $$$ we could not match at the time. For the foreseeable future, we’d like to keep our working environment this way.
- We received a very typical mix of cyber analyst (~66%), engineer (~20%), and military (~10%), with a few other candidate types making up the difference to a full 100%.
- Monthly expected compensation of screened candidates ranged from “no way they could be that low” to “who are they? The CEO of a Fortune 500?”. For a select few candidates who were far below reasonable value or offered to be unpaid as a volunteer, we candidly let them know that if we selected them, we would increase their pay to be better aligned and competitive with the pool.
- There’s nothing you can pay for as worth it as good old-fashioned organic employer brand awareness. We definitely do not have all the tactical components like career pages, company reviews, employee advocacy, and fancy awards - but we are so grateful for the community support of Pulsedive and the quality of talent we can refer/attract. I personally owe deep thanks to the many friends who helped advocate this role and got the word out.
In a startup, every process is filled with lessons learned and ideas for increasing efficiency the next time around. Overall, I’m pleased with our process and looking forward to the next time I can dust off my hiring playbook. I hope seeing some of the raw numbers and specific practices will help you, whichever side you are on in the talent acquisition journey.
Have questions? Feel free to shoot me a message.