How LOYAL AI is harnessing NLP to unlock website archives for content creators

07.01.2022 · By LOYAL AI

One of the biggest challenges facing publishers today is how to access deep archives on a daily basis for internal linking and content optimisation purposes. LOYAL AI is delivering a powerful new solution, enabling content creators to surface highly related articles in seconds with advanced article matching technology.

Delivering advanced website search

At LOYAL, we’re working hard to make life easier for content creators. We know how difficult it can be for writers to hunt for relevant internal links, especially across large publications with years worth of archived content. This could mean trawling through hundreds of articles on a standard website search, without the guarantee that the correct link will even appear.

It can be frustrating work. Often articles are poorly categorised, lacking specific tags and get lost in pages of results. We knew there had to be a more efficient and intelligent way of accessing even the largest of archives in order to save content creators time and deliver real value to publishers of all sizes.

That’s why we’ve spent significant time developing a tool powered by artificial intelligence (AI) that can hyper-focus your website archive search and pinpoint the exact information you’re looking for in a matter of seconds.

But what makes LOYAL so unique?

1. Advanced article matching

Unlike traditional search technology, LOYAL uses natural language processing (NLP)  check out our tip box below for an explainer  to power its search. By analysing large amounts of language data, the search tool is ultimately able to “understand” the contents of a document (in this case an article) so that the most relevant and helpful information can be surfaced with just one click. 


What exactly is natural language processing? Put simply, it’s a technical term for how people train computer systems to process and analyse human language. It’s the magical place where linguistics, computer science and artificial intelligence meet.

“Traditional search engines typically work using just keywords,” explains Leonidas Constantinou, Machine Learning Engineer at LOYAL AI. “But these systems that rely solely on keyword matching have a lot of disadvantages and often don’t yield the best results.”

For example, imagine a writer were to input the keywords { player. football } into a traditional search engine which held the following two documents:

1. {playing, soccer }

2. {play, football }

This specific input would only be matched with the second document, {play, football } and the first document would be disregarded. While we as humans of course know that the words “playing”, “play” and “player” are all linked in some way, it takes a sophisticated NLP system to decipher their connection. This is also true for the words “soccer” and “football”.

But it gets even more complex. Search models also have to be aware of something called polysemy (see definition below). Essentially, to understand the true meaning of a word, it needs to be set in context. However, this requires a high level of linguistic understanding that many search models lack. 


Polysemy definition: the coexistence of many possible meanings for a word or phrase. This can be confusing for traditional search engines and produce misleading (and sometimes humorous) results. 

Take this example for instance: 

  • The newspaper got wet in the rain.
  • The newspaper hired a new editor.


  • Paul was a really bright student.
  • The lights in the room were really bright.

Words that look the same but have a different meaning need to be understood in terms of their relationship to other words. We need to rule out the fact that Paul is emitting light from his forehead.

Language models, also dubbed deep neural networks (algorithms that can identify patterns), can be used to extract specific relationships for each word. These intelligent models are able to extract latent and hidden relationships as well. For example:

  • Antonyms
  • Synonyms
  • Or more complicated relationships beyond their syntax

Using advanced search technology greatly improves the chances of yielding highly relevant results based on the input text.

Therefore, when piecing together important narratives, content creators should be wary of simple keyword-based search models as they can fail to capture crucial sources. However, advancements in technology are helping writers to access information, leaving no stone unturned.

“We currently use the state of the art method for keyword-based matching which is far more advanced than the traditional models,” says Leonidas, who has created an NLP model for LOYAL that accurately ranks each article based on its relevancy.

“We are also constantly experimenting with the latest breakthrough technology in the world of AI and language models in order to extract the best value in our intelligent similarity matching method,” he says. 

2. Search with a word, sentence or entire document

However, what makes LOYAL even more helpful, is the ability to search with a specific word, section of text, or an entire document. By inputting an entire article into the tool, writers can really target their results by ‘feeding’ the neural networks with more information.

Watch this video to see LOYAL in action:

Writers can simply highlight a section of text within their document to either search for news or relevant articles within their own publication’s archive.

3. Quickly understand tone of voice

At LOYAL, we’ve created a simple traffic light system – green, amber and red – to signal whether an article is positive, neutral or negative in its tone. “Sentiment can take many forms but we use AI to extract a number from the article that represents the sentiment of the text,” says Leonidas.

Emotion is becoming a much more important dynamic in how news is produced and consumed. Sentiment analysis is a data mining approach that can help writers understand the ever-increasing volume of online news and the shaping of public opinion.

This is a great way for writers to gain additional insights into an article before they spend time extracting information. Take a tour of our sentiment analysis feature here.


4. View article topics at a glance

To help writers accurately assess any article in seconds, LOYAL displays key article topics for each article within its search panel. This supplies a quick summary of the text and can also help to speed up the tagging of content.


5. Improve your archive search and internal linking in CMS

Searching publishing archives within content management systems such as WordPress can be a challenge for content creators. It’s often limited in the results it returns. If your keyword doesn’t appear in the article title, related articles won’t be retrieved.

LOYAL works differently. Using NLP, the tool analyses the full article for more accurate results and better internal linking opportunities. Writers can quickly identify relevant articles.

Check out this video to see how LOYAL works in CMS.

Read more on why publishers should focus on internal linking to boost SEO

New opportunities for publishers

Renowned writers, editors and brands such as The Week, talkSPORT, Pulse and Thrillist have all taken advantage of the enhanced internal linking and searching benefits of LOYAL. 

“Being able to flick between external and internal content is much faster than our current process of opening and switching tabs where you can lose your train of thought,” says Holden Frith, Digital Director at The Week.

“Internal search and linking is far too often an afterthought,” he says, and publishers are increasingly realising the untapped value held by content archives for content optimisation, boosted SEO, and increased readership. Editorial tools such as LOYAL are opening up exciting new opportunities for publishers.  

For more on how to intelligently unlock your publishing archives with LOYAL check out this blog post.