Appen Crowdsourcing produces invaluable crowd sourced data

Kerri Reynolds knows a thing or two about hiring. As Appen’s new Senior Vice President of Human Resources (HR) and Crowdsourcing, she’s been a professional talent acquisition recruiter for 20 years for Fortune 500 companies including Citibank, MasterCard and Microsoft. Joining Appen in March of this year, Kerri has jumped right into the challenges of recruiting the crowds we rely on to provide solutions to clients in web search evaluation, social media evaluation and linguistics. Asked about her decision to take on this new role, she said, “Appen is on such a tremendous growth path. I wanted to be a part of that.” Kerri is continuing the great work our recruiting team has been doing for years, as FlexJobs recently named Appen as the number one provider of remote jobs in its annual list of the 100 Top Companies to Watch for Telecommuting and Remote Jobs. As the winner for four years running, we’re proud of the high-quality, flexible work opportunities we continue provide to people all over the world. Crowdsourcing: What Is It? The HR side of Kerri’s role is fairly traditional. Crowdsourcing is anything but. You may be familiar with the term from fundraising with sites like GoFundMe and Kickstarter, for idea contests, or web-based projects like Wikipedia. Appen uses crowds to perform big projects in short periods of time. Part of Kerri’s job is overseeing the recruitment of those crowds. “We’ll have a client who needs 500 people to work on project X,” she explains. “My role is to coordinate the recruiting team so we can quickly identify the right people for that project, who may be anywhere in the world, and get them onboard and available for work quickly.” For Kerri, building the crowds is both the biggest challenge and the most rewarding part of the job. “It’s exciting to find 1,000 people around the globe who want to work 20 hours a week in their local language to make the web better. It takes a compelling value proposition for workers and good organization within our team. Ultimately, it helps our clients build better products.” About Appen Crowdsourcing Appen uses crowds for a number of different types of work, including web search evaluation, social media evaluation and linguistic resources. Web search evaluators and social media evaluators look at data that’s going to live on the web and rate it for relevance. Workers in these roles help our clients create better search experiences, so that when people use a search engine to find something in particular, they get the best results. “For example,” says Kerri, “say you’re looking for local news. You type that query into a search engine and you get articles in order of most to least relevant. The search engine returns what you’re looking for because someone rated data that improved the relevance of the search. It might have been done by an Appen crowd worker.” For these positions, Kerri and her team are always on the lookout for highly detailed individuals: people who can look at an image or read something, quickly deduce what’s going on, and annotate it. The right candidates must also be technically comfortable with social media and other platforms. What is Crowd Sourced Data? You’ve just learned how Appen managed search and social media evaluators annotate and evaluate data, rating it for relevance, the data they produce is a great example of crowd sourced data. Appen’s Language Resources team helps clients improve their voice-recognition systems. If a company wants to launch its product in a new market—let’s say Brazil—then it may come to Appen and request a number of scripts read in Brazilian Portuguese. Our job is to find multiple native speakers who can read those scripts in natural language. The client then feeds this crowd sourced data to its computer, so when people speak to it in Brazilian Portuguese, the computer understands what they’re saying. Why does this work require multiple people and ideally “crowd sourced”? Because in every language, there are subtle differences among speakers, such as regional dialects. “For example,” Kerri explains, “I have a New York accent, so the computer in my home or car needs to recognize that when I saw ‘water,’ that’s the same word as when someone from Kansas says it, and when someone from the U.K. or any other country says it. By sourcing a number of different speakers, we teach voice-recognition systems that all those sounds mean the same thing.” Recruiting globally, at scale Traditionally, recruiters focus on looking for one person to fill a full time role for an indefinite amount of time. A recruiter creates a requisition, searches for candidates, manages the interview process and coordinates hiring. An Appen crowdsourcing request looks more like asking 500 people to work 20 hours a week for one month only, starting as soon as possible. “To look for a crowd,” Kerri explains, “you need scale. A post on the company website or Monster.com just doesn’t cut it. We need to reach 100,000 people to hire 10,000. We rely on social media advertising—blanketing social media actually—so we can reach people from multiple angles.” As a crowd of 2,000 people might be spread across 16 different countries, the on boarding takes place online. Luckily, Appen’s many locations around the world make this easier. What’s in it for the crowd? For job seekers, being part of Appen crowdsourcing has many benefits: flexible hours, competitive pay, home-based work and the opportunity to help improve the quality of web content. It’s a combination that works for a broad spectrum of people. “We find that students, stay-at-home moms, and seniors with computer skills and extra time on their hands match up well with crowd work,” says Kerri. “We also work with a lot of military spouses or other people who have moved to foreign countries and are looking for work.” One longtime crowd worker, a student, worked for Appen throughout her time in graduate school. The income helped cover the basics while she achieved her master’s degree. Another success story is a stay-at-home mom whose husband lost his job. She worked for Appen to help make ends meet while still staying home with her kids during the transition time while her husband looked for and landed his next job. Responding to demand Crowds are a great solution for evaluating technology because they can quickly respond to the fast pace of change in the industry. For example, artificial intelligence (AI) is becoming a larger part of our lives, appearing in everything from cars to phones to in-home personal assistants. Still in a relatively early stage, AI is constantly improving, but it relies on huge amounts of training data to get better. “Behind all this AI, humans still need to touch the data,” says Kerri. “We still need human power to teach computers the subtleties of identifying data, language, dialects, etc.” Recently, Appen’s Language Resources team has expanded to meet the increased demand from AI solutions, including building linguist crowds and a new office in Detroit to better support our work for clients making the voice-operated infotainment units in new automobiles. Our ability to quickly scale up or shift priorities is in part due to our smart use of crowds. For Kerri, one of the most exciting things about working for Appen is that there’s new technology coming out all the time—and a lot of it requires data analytics and data tagging. “At Appen,” she says, “every day holds something new. With increasing industry demand, Appen is growing, and we’re always looking for more people to join us. We’ll continue to offer people excellent options for work they can fit into their lives to make money for their families—and help Appen move the tech world forward.”

AI Requires a Human Touch: How Appen Recruits Crowds to Improve Technology

More Articles Like This

Blog

The Impending Data Crisis in the AI Economy

Blog

Deciphering AI from Human Generated Text: The Behavioral Approach

Blog

Building AI We Can Trust

Blog

Appen and the UNGC: Defining Sustainability and Ethics in the AI Era

Blog

How the Human Element Balances AI and Contributor Efforts for Optimal Outcomes

Blog

Appen's Benchmarking Solution: Confidently Choosing the Right LLM for Your Application

Request a Consult

Request a consult