Beyond experts: jobs, tasks, and skills for a data driven Future of Work

The World Economic Forum (WEF) is upon us this week, and the Future of Work is one of its key themes. This is a good opportunity to catch up on the trends unfolding in this domain right now, and to ponder on the insights of the people taking note and shaping this discussion.

Automation and AI is part of this discussion as well, with the jury still out as to how exactly this will shape labor, workforce dynamics, and workplace transformation among others. Based on the WEF’s latest report on the Future of Jobs, we highlight the major forces at play today. We discuss how these effect the technology behind the job market with Panos Alexopoulos, Head of Ontology at Textkernel, a Careerbuilder company. 

Textkernel powers Careerbuilder, one of the world’s top employment services. Textkernel delivers advanced features such as multilingual semantic search and matching technologies for Careerbuilder. Alexopoulos, an expert in knowledge representation and reasoning, is leading a team building
knowledge graphs
to match job seekers and opportunities at Textkernel. We discussed how trends in employment translate to requirements for his work..

Automation versus augmentation, job titles, tasks, and skills

One of the key takeaways from WEF’s report is that automation continues its stride, with the advent of AI meaning it can be applied to an increasing number of tasks. This, however, is a crucial observation: tasks do not necessarily equal jobs.

The job of a personal assistant may be a good example. Tasks such as typing and dealing with communication (via mail, phone, or fax) used to be a core part of this job’s function. Skills associated with these tasks have evolved (think keyboards and recipients versus typewriters and envelopes), while it can be foreseen that they may be almost fully automated as technology in areas such as transcribing for example progresses.

We now have word processors, email and voice recognition, yet personal assistants still exist. They are more productive in their tasks, which have been updated with new skills. The tasks are partially automated, and could potentially be fully automated in the near future. Personal assistants have been augmented, not fully automated. They may move up the stack by taking on more value add tasks. 

This is why the WEF highlights a net positive outlook for jobs. This also goes to show that there is some sort of association between jobs, tasks and skills in the real world. But what would be needed in order to deal with the evolution of definitions and requirements for employment? Alexopoulos noted that the tasks and skills a particular profession involves are very important aspects of the profession’s meaning:

“There are practically no job vacancies without description of tasks and skills. This is in line with knowledge representation and semantic modeling practice, where a concept’s meaning is formally defined not merely via its name(s), but also via its relations with other concepts. 

If you want your systems, but also your non-expert users, to understand and reason with employment concepts, you need to somehow capture the associations between job titles, tasks, skills and other concepts (e.g. qualifications)”.

Documenting job functions, tasks, and related skills: is there a standard for that?

It’s clear therefore that documenting job functions, tasks, and related skills is an essential requirement. But how is this done today? When discussing with Alexopoulos, we initially touched upon 2 approaches in this area, ESCO in the EU and O*NET in the US. But as Alexopoulos pointed out, there are more approaches than ESCO and O*NET as most EU countries have their own standard: 

“This is an indication of how difficult the task of documenting job functions, tasks, and related skills is. There is a lot of of diversity (linguistic, cultural, economic) that an aspiring global standard needs to capture. As a simple example, consider the pretty different formal qualifications and legal knowledge a lawyer needs to have in the US and in the UK.

The biggest problem is usually though with the informal, undocumented differences in non-regulated professions. For example, there are no ‘lorry drivers ‘ in the US, there are ‘truck drivers”http://www.zdnet.com/”.

Alexopoulos believes that standards like ESCO and O*NET are pretty good when it comes to defining high level concepts and categories, as well domains that do not change very fast. But he noted that it can be hard for such standards to keep up with concepts and associations that change very frequently. For a job like data scientist, the top 10 tools today are not the same as 5 years ago, as new technologies and tools emerge.

Another limitation of standards, which Alexopoulos thinks is inevitable, is that they are not necessarily usable for all types of applications. Alexopoulos noted for example that ESCO is pretty good for reference and navigation, but not so much for Natural Language Processing tasks like skill tagging, as the labels it uses for concepts are too verbose.

Beyond experts: going data-driven

While documenting jobs, tasks and skills is essential, it seems current standards for doing this are not entirely up to the task. What can be done to mitigate their shortcomings? Alexopoulos believes that the creators of such standards should be more data-driven, and not merely rely on a handful of “experts” to suggest what skills and activities a profession has or needs to have.

The “growing skills instability”, as the WEF calls it, does pose a challenge for standards. But emerging in demand technology and the new roles and capabilities it brings may help deal with it. The fact that Careerbuilder relies on knowledge representation techniques and knowledge graphs to power its matching speaks volumes on the capabilities of this technology. 

But it’s also a fact that such approaches are hard to maintain at scale, especially in domains with rapid “concept drift”, as per the knowledge representation terminology. Things change, to put it simply. This is known to anyone who ever tried to keep track of a sufficiently large domain over time. As Alexopoulos pointed out, relying on experts can only take things so far. So how can curated knowledge representation efforts cope?

Becoming data driven may well mean adopting machine learning approaches in addition to expert curators. We have seen for example how this can be applied to build knowledge graphs at web scale. Granted, this is no easy feat. Not just because of the infrastructure required to harvest and process data at web scale, but also because of the complexity.

When talking about jobs, we are pretty much talking about the entire spectrum of human activity and knowledge, past present and future. Any arcane domain or task may be at some point associated with economic activity. While applying machine learning to specific domains can perform well, the more the domain widens, and the more domains to keep track of, the harder things get.

Another approach that can be used to bridge the gap between experts and the real world is going agile, and involving more actors in the process. The agile approach is taking the world by storm and this, too, is bound to have a profound effect on the future of work. More on this in the second part on the interplay between data, automation, and the future of work tomorrow on ZDNet Big on Data.