If Amazon’s Tool Could Discriminate, Could Yours?

Yesterday, Reuters reported that Amazon created a recruiting engine using artificial intelligence.  This isn’t news.  Amazon is a leader in automation, so it makes sense that the retail giant would try automation in their own recruiting processes to try to quickly find the “best” candidates.  Yet, Amazon’s tool had a big problem – it didn’t like women.

As the article describes, “Everyone wanted this holy grail,” one of the people said. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”  Who doesn’t want this?  To make hiring faster and easier?  Currently, there are hundreds of AI tools available to human resources – many of them in the recruiting space – that promise to do these things for you.  But if Amazon found problems, what about those tools?

Amazon’s tool used a 10-year look back of existing employees (largely male-dominated).  The tool then could rank applicants based on what it learned makes a good Amazonian.  Based on its own analysis, the tool learned that male candidates were preferred over female candidates in a mixture of words that appear on applications, like “women’s,” experience, job requirements, and potentially proxies for gender.  While Amazon tried to solve for this problem – making “women’s” a neutral word so the tool did not reduce the applicant’s rank – the results of the tool still had a negative impact on women.  So, in 2015, Amazon abandoned the tool.  Good for Amazon.  This is the right thing to do.  But again, there are hundreds of other AI tools out there.

At this year’s HR Tech Conference in Las Vegas, my friend Heather Bussing and I presented on this very topic.  We spoke about how AI can both amplify and reduce bias. Here are a few of the highlights:

  • We know that AI is biased because people are biased.
  • We know the sources of the bias include the data we use to teach the AI, the programming itself, the design of the tool, and people who create the tool.
  • Employers have to be vigilant with their tools.  We have to test for bias and retest and retest (and retest) for bias in our tools.
  • Employers – not the AI – are ultimately responsible for the results of the tool, because even if we follow the output of the tool, the employer is making the ultimate employment decision.

It is very possible, even probable, that the tools out there on the market have bias in them.  Employers can’t simply rely on a vendor’s salesperson’s enthusiastic declarations that the tool eliminates bias.  Instead, employers should assume bias plays a factor and look at their tool with a critical eye and try to solve for the problem ourselves.

I applaud Amazon for doing the right thing here, including testing its tool, reviewing the results, and abandoning the tool when it became clear that its bias played a part the results.  This isn’t easy for every employer.  And, not every employer is going to have the resources to do this.  This is why employers have to be vigilant and hold their vendors accountable for helping us make sure bias isn’t affecting our decisions even when using an AI tool.  Because ultimately, the employer could be liable for the discrimination that the tools aid.


Photo by Kevin Ku on Unsplash

Die Annual Performance Review Die

Client calls.  Asks if they can fire Jerry for performance reasons.  The first (seriously, the very first) question I ask is, “what do Jerry’s performance reviews say?”  Experience has taught me that performance-related terminations usually have a homegrown enemy – the employee’s previous annual performance reviews.  What if we could eliminate the enemy by doing it better?

No one likes performance reviews.  Employees lose sleep the night before a review meeting.  Managers hate completing all the forms and fear having uncomfortable conversations.  HR turns into nagging mother-in-law types trying to track down managers to get all the forms turned in so that performance increases can be made.  No one likes this.

Performance reviews are rarely done well.  Most typically, the reviews are so vague they are meaningless.  They focus only on recent events and not performance over the entire year.  They are chockfull of bias.  Sometimes, a manager pretends he lives in Lake Wobegon where all the employees are above average.  Because we in HR are focused on handling the next fire, we don’t have time to push back on managers who do not do performance management well.  So, a poorly completed review gets stuck in a personnel file until I ask about it when the client wants to terminate.

Even when the termination is completely warranted and lawful, it’s the performance review that hurts.  The termination is going to have to get explained.  I’m confident that I am not the only employment attorney stuck explaining why an employee was terminated for bad performance just weeks after a positive review.  (We attorneys should form a secret society complete with a secret handshake.)  Our explanation is often couched in terms of a rapid performance decline as explained by a manager who “wanted to be nice” in the review but had observed poor performance that resulted in a lost customer, order, and so on.  The explanation by both the attorney and the manager is expensive for the company.

These are just a few of the reasons I want the annual performance review to die.  I’m not advocating for the end of performance management – quite to opposite.  I want more frequent, meaningful reviews for everyone.  Here’s my wishlist:

  • Conversation coaching.  Managers need to have difficult conversations with employees about performance.  Most managers, and particularly new managers, have not learned how to have these difficult conversations.  HR pros are conversation coaches, so we need to coach our managers on how to have these conversations.  Or, we need to get our managers the training and skills necessary.
  • Frequent discussions.  I love one-on-ones when they’re done right.  Brief meetings that discuss how projects are progressing that also discuss how the employee is doing are vital to successful businesses.  With this, managers get a sense of what roadblocks they can remove, and employees get critical feedback on how to do better.
  • Transparency.  People need to know how they’re doing.  Managers need to tell them.  Use examples.  Explain how things can improve.  Show.  If employees know where they stand, they may be able to understand why you’re firing them and not believe it is for some unlawful reason.
  • Recognize.  It isn’t just poor performance that needs to see the light of day.  Good performance does too.  Managers need to know how to champion those performers with potential as well as coaching those who just haven’t meet expectation quite yet.
  • Documents.  (Insert collective reader sigh here.)  Yes, feedback discussions should be documented.  I don’t care you document provided you document and I can get it later when we need it.  You can use the functionality of your HCM or you can have managers email themselves brief synopsis of each conversation.  With the conversation coaching, coach managers how to document as well, including how to remove references to protected class status, leave use, or other items that could get an organization in trouble.

Employees deserve to know how they are doing.  More importantly, they want to know how they are doing.  That’s what a great performance management process can do – get employees what information they need to do their jobs well so we can do our business well.


Photo by Tim Gouw on Unsplash



HR Tech’s Adverse Problem

While I totally loitered at the Society for Industrial and Organizational Psychology Conference (I was a presenter, just failed to register – oops), I’d thought a post on what we talked about yesterday and a bit about what’s happening at the University of Minnesota’s HR Tomorrow Conference today: adverse impact, why it’s important, and why you should care.

Adverse impact (known as “disparate impact” by the lawyers) is when groups of individuals described by a particular characteristic is negatively affected by an employer’s decision, selection tool, or policy when that decision, tool, or policy is neutral on its face or does not intend to actually have a negative impact.  For example, if an employer uses a psychological test that filters out African Americans, the test would have an adverse/disparate impact on African Americans.

The concept of disparate impact has been around for a long time.  The United States Supreme Court in Griggs v. Duke Power formally recognized the claim.  Since that time, the law has been debating many aspects of the claim, including what statistical models to use, does the doctrine apply if the rule intends to discriminate, how does impact different from treatment, and will the doctrine apply to all the HR technology out there.  While this post could go on-and-on about all of these questions, this last piece is really important for HR tech buyers, and the answer is probably.

We already know that lots of HR technology vendors, including the fancy-dancy stuff like artificial intelligence, machine learning, algorithms, etc., market their products as the only way to find the best candidates, identify problem employees, and make all your dreams come true.  When these technologies are used, their use could create a disparate impact.  How do we know?  Because we’ve already seen how these technologies discriminate outside the world of HR – see photo ID that classifies African Americans as gorillas, recidivism tools that increase prison terms for African Americans, etc., so it is highly likely that they could operate the same way when it comes to HR tech.  Arguably, HR tech has the potential to greatly impact because the decisions HR makes affect individual’s livelihood.

So what should we do about diverse impact?  While there are many, many things we need to do to limit the potential that the HR tech we use doesn’t discriminate, we should start with two things.  First, we have to know how the technology works and the data it uses to make recommendations.  This requires vendors to be open and honest with us, lose the marketing gloss, and really explain their products. Can they explain how the tech works?  Can they explain how the tech works on our organization’s data?  Could the data have bias baked in?  (The answer to this last one is probably yes, especially if we’re looking at hiring or performance data.  There’s just no escaping it.)  When vendors are transparent and honest about these issues, we can take more steps to mitigate any disparate impact the tech might have.

Second, we need to test and test and test to see if the tech creates the disparate impact.  Lawyers and data scientists talk about validation as the test.  For lawyers, validation means under the Uniform Guidelines for Employee Selection Procedures.  For data scientists, validation means how strong the correlations are statistically.  This definitional problem causes more debate and potential confusion.  So, we need to find vendors who understand, appreciate, and can articulate validation under both tests.  Because the HR tech world is a bit like the wild, wild west, it’s hard to find them. (Trust me, they’re out there.  I’ve probably met them or at least brow-beat them from a distance on this very issue.)

All that said, I want HR to understand and appreciate that these issues could exist and start playing an active part in fixing these issues.  While I’d love for everyone to trust each other, placing blind faith in a vendor is not in our organizations’ best interest.  Holding people accountable is one of the strengths in HR.  We should use it here too.

One final note, I love this stuff.  This tech is going to revolutionize how we do business.  I just want to do it in such a way that doesn’t create that much risk for our businesses.  Remember my pledge?


Photo by Patrick Lindenberg on Unsplash

Happy Birthday, tHRive!

Today is a big day!  Today, tHRive Law & Consulting turns one.  In just the past year:

Human resources and employment law are ever-changing and exciting.  Our work touches nearly everyone, making it incredibly meaningful and challenging.  This is why I love it.  I can’t think of another area of business or law I’d rather be in.

tHRive Law & Consulting made it through one of the most significant milestones of any start-up – the first year.  I could not have done it without the support of so many and the confidence of my HR tribe.  For that, I am eternally grateful.  Thank you!

Now, onto the challenges of year two!

Photo by Markus Spiske on Unsplash

Performance Data Based Analytics

How can you do big HR data analytics when you eliminate performance rating?

A well-known people analytics person posted this question on LinkedIn a week ago.  The discussion that followed was fascinating.  The question took my breath away for a couple of reasons: (1) performance ratings are one of the most bias-ridden data points on any employee, so it makes me queasy to think such data would be relied upon heavily in an analytic, and (2) so much more data exists that can provide better insights.

Let’s take the biased data first.  While I am no Marcus Buckingham devotee, his piece for Harvard Business Review back in February 2015 encapsulates the idea that HR data – particularly performance data – is bad data.  By in large, managers rate employee performance on how they would perform the job, not on the actual performance of the individual employee.  If, as Mr. Buckingham points out, 61% of a rating is really a rating of the rater, then ratings aren’t that useful in making people decisions.  (In a post for another time, performance management doesn’t even need ratings.)  This bad data makes for bad analytic results.

This is furthered by oodles of studies that show unconscious bias is baked into performance data.  There is no escaping it.  The unconscious bias of managers – which is not to shame managers, we all have bias – is active in performance reviews and data.  If women and minorities are rated more harshly because of the bias, then an analytic tool that relies on performance data also contains the bias in those harsher critiques and could perpetuate or exacerbate discrimination.  This can occur even when we remove demographic data through data proxies.  Who wants biased data in their analytic tool?

Next, the idea that HR analytics must rely on performance data misses the plethora of other data that we could use to make people decisions.  Here are but a few examples:

  • Great leaders have connections throughout companies. We can find out who has developed a network within a company by reviewing email connections, social media connections, and other network analysis.
  • Internal threats sometimes start with emailing themselves information on personal email accounts. Monitoring access (authorized and unauthorized) along with other retention analysis can help identify who could be stealing our trade secrets or confidential information to take to a competitor.
  • We can better predict how to scale hiring and what skill sets are needed based upon productivity and sales projections.
  • Things like weather, productivity, date, and time can all be factors in safety incidents. If we analyzed these items, we could develop a work schedule that reduces worker injuries.

These examples show employers could do better without touching the bad and biased performance data.  If we didn’t include biased and bad data, would it be the end of people analytic tools?  I emphatically answer “no.”  We could do even better without it.


Image by Markus Spiske available at unsplash.com



Those Pesky Correlations

Last week, the ABA held its Section of Labor and Employment Law National Symposium on Technology in Labor and Employment Law.  This is one of my favorite conferences, because I get to geek out with some of the most forward-thinking employment lawyers in the country.  This year was no different.

This year, we had two separate sessions on people analytics.  Analytics has been a hot topic for HR for the last five years (at least), and many employment attorneys are trying to play catch-up.  Using algorithms, artificial intelligence, and at times machine learning, analytics crunches data (employer, employee, social media, and/or public data) to find correlations that assist employers in making decisions.  These decisions can range from finding the best candidate to unmasking the employee stealing trade secrets.  Analytics are really, really cool!  But beware, problems exist.

A big one is discrimination.  We’ve known that analytics can be discriminatory for a while now.  We’ve seen discriminatory results in analytics in the justice system, advertising, and many, many others.  Because we’ve seen discrimination elsewhere, it could happen in people analytics too.  If it does, how will the law handle it?  Will a judge review an analytics case like she would a neutral policy that had a discriminatory result when used?  Or will a judge review an employment decision on an individual basis?  As people analytics develops, employment lawyers are finding themselves divided on how the law will deal with analytics that result in discrimination.

On one side, there are data scientists and a few management-side attorneys.  They think that when the analytics draws a correlation, that correlation is statistically strong, meaning the correlation has a strong relationship to the job or job duties.  For example, coders who visit certain manga sites are better coders than those that don’t.  The statistics show this, so the logic is that you should only hire coders who visit manga sites.  But what if black coders don’t go to manga sites, and now your analytic tool is weeding them out.  This certainly looks like discrimination.  But the logic of data scientists that the statistics support the idea that good coding and manga sites are linked and therefore job-related and a business necessity under Title VII.  People analytic vendors love this.

On the other side are some attorneys (and some industrial psychologists) who believe that the statistics alone will not be sufficient to prevail under the law – an employer must show more than just the statistics to overcome Title VII’s job related and business necessity requirements. Professor Pauline Kim of the University of Washington St. Louis School of Law argues that the correlations need to be both statistically valid and “substantively meaningful.”  She argues (and I agree) that there needs to be some connection to the job that’s more than just math.  If a coder is coding for a manga site, then the criteria that the coder visit manga sites make sense – it’s substantively meaningful.  If coding for a workplace software company, it wouldn’t.  This adds a “smell test” to the statistics that a jury can understand and hold on to.  People analytics vendors don’t love this as much because it means they would have to validate their tools using more than just statistics.

The debate at the conference was lively.  We just don’t know what will happen and what theory will prevail.  The EEOC is certainly paying close attention to people analytics.  Last October, the agency held a public meeting on the subject and heard from many different stakeholders on the subject.  Acting Chair Victoria Lipnic is very interested in where analytics is headed.  So am I.