Unlocking the ROI of Learning

Or how to free your data from the LMS

Relevant to: HR, L&D, Operations, Finance

You’re under pressure to show the impact of learning. You need to prove that training links to performance, that learning programmes improve retention, and that investment in learning is moving the profitability needle.

But when you go looking for answers, all roads lead to your learning management system (LMS) — and stop there.

You know the data is in there somewhere. Completion rates, assessment scores, time spent learning, department-level engagement — all the raw material for powerful insights. But what you get instead are clunky exports, static reports, and dashboards that don’t speak the language of the business.

If this sounds familiar, you're not alone.

You’ve got the data!

You know you’ve got the data, that’s why you have an LMS. The bad news is that it’s not always easy to get at – particularly if your LMS is software as a service (SaaS).

If you have found your valuable learning data locked in your LMS, I feel your pain. This wasn’t what you were expecting.

Using your data

There are different ways to use data stored in your LMS. For many, the standard reports are more than enough. But for those that want to analyse learning data together with other data, or do more in-depth analysis, you have to get data out of your LMS and into a reporting data store. Here’s how that might work:

1. API extraction – most modern LMS platforms provide RESTful APIs. With proper authentication (usually via OAuth 2.0 or an API key) you can programmatically extract learning records.
2. Data pipeline & transformation – Data is ingested into a reporting database such as Azure SQL Database, where it's cleaned, normalized, and enriched with metadata.
3. Semantic modelling – Using tools like Power BI, you can then build a semantic layer that defines business terms — e.g., "active learner", "average time to completion", "learning impact score".
4. Dynamic dashboards – These models power interactive visuals, filterable by time, team, location, or training programme — and update in real time or on a schedule.

If you are wondering why you need to store, check and clean data in yet another data store, I talk about that here: Can you trust your data? The bottom line is that your data needs to be clean, up to date, and readily available for analytics purposes. 

Power BI is L&D’s new best friend

With this approach, you’re no longer limited by your LMS’s front end. You get full control of your learning data — and the power to connect it to performance or finance data for deeper insights.

So, although you may sometimes feel that your data is locked in your LMS, there are ways to get at it and analyse it in friendly tools like Power BI.

Here at Anatec AI we’ve been working with data, interfaces, APIs, and learning systems for many years. So, we are well placed to help if you need it. And we can help with Power BI dashboards, scorecards, DAX queries and semantic models. There’s nothing we’d like more. 

If you have a question or want to chat about any challenges you’re facing, get in touch.

Keywords: learning analytics, data analytics, learning management system, LMS, Power BI, L&D dashboard, L&D scorecard

Can you trust your data?

If you’re relying on data to make decisions, here’s a question for you:

Can you actually trust your data?

Bad data leads to bad decisions. And if you’ve ever tried to build a Power BI dashboard using unprepared data… you know what I'm talking about. That’s because good analytics starts before the dashboard—with trustworthy data.

So, let’s talk about what makes data trustworthy—and how to get there.

The 3 C’s of trustworthy data

To be useful, your data needs to be:

• Correct

• Current

• Constant

These three qualities are the backbone of any solid reporting system. Miss one, and your insights could be misleading at best—or flat-out wrong.

1. Correct data: accuracy isn’t optional

Data errors creep in more easily than you think. A common name mix-up might credit someone with attending a course they never showed up for. Or a stock item might never be recorded because “we were going to use it straight away.”

Sound familiar?

Then you’ve got things like:

• Null values

• Duplicates

• Outliers

• Inconsistent fields between systems

All of these distort the truth your analytics are supposed to reveal. Cleaning and validating your data isn't optional—it's foundational.

Question for you:

What’s the most unexpected data error you've ever uncovered?

2. Current data: how fresh is “fresh enough”?

Everyone has a different definition of “up-to-date.”

• For a factory floor, real-time data might be essential.

• For HR reports, yesterday’s numbers might do just fine.

But what matters most is transparency—do you know how current your data is, and can you trust that timestamp?

3. Constant data: reliable, available, and secure

Once your data is cleaned and verified, you need to keep it:

• Securely stored

• Regularly backed up

• Available wherever it’s needed

You don’t want your cleaned dataset disappearing on a lost laptop or overwritten by mistake. Constancy means your data is dependable and accessible, day in and day out.

Choosing the right data platform: why Azure?

The Azure data platform gives you flexible, scalable ways to store your analytics-ready data, depending on your requirements:

Azure SQL Database

• Great for datasets up to a few terabytes (TB)

• Geo-redundant and cost-effective

• Easily scalable up or down

• Supports Medallion architecture using schemas or databases

(If you’re curious about that approach, I wrote more about it here.)

Microsoft Fabric

• Ideal for high-performance analytics at scale

• Better suited for large volumes of data

• Higher cost, but better performance

• Also supports the Medallion architecture.

Once your data is in the cloud, everything gets easier—from sharing semantic models to boosting Power BI performance.

Don’t skip the foundations

Data visualisation tools like Power BI are only as strong as the data underneath them. Trustworthy data isn't just clean—it's correct, current, and constant.

So, here's a challenge:

What’s your biggest headache when it comes to data quality or reporting?

Drop it in the comments—I’d love to hear what you're wrestling with (and maybe swap ideas on how to fix it). And as always, if you want to talk about data quality, you can get in touch here.

Anatec AI has worked with data quality issues for many years. We focus on helping companies make better use of their data to improve their performance and resilience.

Key words: reporting, analytics, data quality, Power BI, Microsoft Fabric, dashboard design

Learning data is valuable data

Learning data may be the most underused strategic asset in your company. The world’s most powerful companies are increasingly evidence-led, yet learning and development (L&D) is often under-represented when it comes to data analytics. Right now, it has never been more important to nurture and retain good people. AI tools such as ChatGPT can assist skilled and knowledgeable people; but they don’t replace them. 

Blended learning - messy data

The move from classroom training to on-line or blended learning has reduced costs and facilitates less time away from work. But it has made learning a lot harder to track. Micro-learning results can be difficult to evaluate when you want a strategic view of your learning.  E-learning, YouTube, and MOOC courses are all more complex than pure classroom training when it comes to managing data. 

The rise of the learning management system

Learning management systems (LMS) make it possible to centralise learning data. An LMS not only enables you to distribute your own learning content but also track other learning in different formats. The only challenge is to turn this valuable asset into actionable insights. 

LMS data and Power BI 

Whilst every LMS has a variety of reports, there’s no replacement for the ability to interrogate data in analytics tools such as Microsoft Power BI. The ability to visualise the data in different ways, add multiple “slicers”, and use AI to find patterns and trends is tremendously powerful. Plus, you can match your learning data with data from other sources such as performance data, HR retention data etc. This flexibility can unlock real insight into what your learning programmes are achieving. Power BI can help you to make sense of a more complicated world of learning, massively improving your chances of finding patterns amongst the noise. 

How Anatec AI can help

If you want to do more with your LMS data and are stuck somewhere along the way – we can help. There are multiple ways of getting data from your LMS and into Power BI, depending on your needs. We can design data interfaces and help you choose the right way to store reporting data. We build semantic models that focus on your business needs and can help you make better use of Power BI AI capabilities.

Whether you’ve got a simple question, or an analytics project you need help with, get in touch to see if we can help. We’d love to hear from you. 

Anatec AI has worked in the learning and development space for many years. We designed and built two custom learning systems before LMS’s were even a thing. Now we focus on helping companies make better use of their data to improve their performance and resilience.

Key words: learning analytics, LMS data, Power BI for L&D, evaluating learning impact

Real-time intelligence: why speed matters more than ever

How real-time analytics is reshaping decision-making, competitiveness, and customer experience

Modern tools like Microsoft Power BI have revolutionised the way we analyse and visualise data. So much so that we’ve almost forgotten that we are looking at historical data that can be days, weeks, or even months old. After years of having no alternative, we’ve become desensitised to the delay between data collection and making sense of what it means.

But in today’s fast-paced and data-driven world, decisions are only as good as the data they are based on. That means that reducing data latency is more important than ever. When it comes to data, speed really does matter.

Having real-time or near real-time data has become a source of competitive advantage, a way of improving products and services, and a way of staying relevant. Data latency is no longer just an IT issue, it is now firmly on the boardroom agenda.

One company that has transformed its product offering with real-time analytics is The New York Times. By migrating to the cloud, and getting real-time data about how their readers behave, they have improved their content and tailored it to user preferences. The result has been greater reader loyalty and greater relevance in an industry that has had to make huge adjustments from print to digital. Whilst the New York Times is a multi-billion dollar business, the principles they are using are universal.

In the past there were good reasons for analysing data in batches: processing streaming data was complex and expensive, in memory storage didn’t exist, and cloud computing was less mature. But these technical barriers have now gone. And new technologies like Microsoft Fabric make it much easier for businesses to benefit from real-time data analytics. Microsoft Azure enables businesses to migrate to the cloud incrementally, delivering business value at every step.

If you’re exploring how to reduce data latency in your business — or how to build real-time dashboards from streaming data — we’d love to help. We are experienced developers for the Azure data platform and can turn ideas into robust data solutions for a range of needs. Why not get in touch for a no-obligation exploratory chat? It can only speed things up ....

How to use Microsoft Power BI semantic models to analyse data from multiple sources

Unlock deeper data insights with Microsoft Power BI semantic models

Microsoft Power BI is a game-changer when it comes to analysing departmental data. One of its great strengths is that it enables you to analyse data from more than one source. That means you can bring in data from say an Excel spreadsheet, an Access database and a line of business application, and analyse it as if it were just one data file. 

Working with more than one data source enables you to unlock more value from your data. It also enables you to use powerful AI visuals such as Power BI’s Key Influencers which needs a wide range of different attributes to give good results. 

But having several data sources can make the data more complicated to work with. Common problems include not being able to get Power BI to “see” parts of your data, or selection visuals not working as you expect. 

Power BI semantic models unlock more value from your data

A semantic model, or more simply a data model, puts data into a format that makes business sense. It is a business-friendly layer between the raw data and the report visuals. 

For example, if you are interested in sales trends, it is likely to want to know sales per day rather than deal with thousands of individual transactions. Creating a DAX measure called “DaySales” within the model makes the data easier to work with, and improves accuracy. You might also want to view “DaySales” by attributes from other files, so the semantic model needs to include those relationships.

Relationships may be one to many, or many to many, and the key to a good semantic model is how best to handle these relationships to enable meaningful analysis of the data. Data in a semantic model is often in a star schema format, with one or more fact tables, and several dimension tables that provide context to the data you want to analyse. Modelling data using a star schema provides a wide range of ways of analysing key metrics. 

So how do you go about creating a semantic model? 

Semantic models step-by-step

Creating a semantic model is step-by-step process, which needs to be done in the right order: 

1. Understand the business needs 

2. Clean and transform data

3. Create relationships, measures, calculated columns, and hierarchies 

4. Add security restrictions

5. Test and optimize.

The first, and arguably most important step, is to understand the business needs. Who will be working with the data, and why? What are they trying to achieve? Who will view the reports or use the analysis? What restrictions should be included, for example should some people see only part of the data set? Analysing the business needs provides a wish-list of requirements that can then be used to create the semantic model. You may be tempted to skip this step, feeling that you know your own business. Although that may be true, analysing your requirements for data analysis is always worth the effort.

Secondly, the data must be clean. This means that it must have the right data types, duplicates are removed, and decisions made about any data that is missing or obviously wrong. As clever as Power BI is, it is only as good as the data it is given.

Thirdly, the semantic model is created by joining data tables to create a star schema, and adding DAX measures, aggregates, and additional columns. Columns can be renamed or hidden to make the data easier to work with. Hierarchies can be created to make common tasks easier, such as managing dates. 

After the semantic model has been created, security such as row-level security can be added if needed. 

Finally, and crucially, the model needs to be tested to ensure it provides the expected results.

As with many things related to data, the order in which you do things is vital. Trying to work with raw data that hasn’t been cleaned will not produce good business results. Equally, providing high quality data that doesn’t make business sense will not produce good results either.

How to create the right semantic model for your needs

Power BI offers excellent visualization and analysis capabilities, making it the go-to tool for departments that want to make better use of their data. With the right semantic model, it’s easy to ask questions, dig deeper, and use AI to analyse your business data. Our step-by-step process allows you to figure out what you need from your data and assess what additional steps you need to optimize your reports. And if you want to understand more about why star schemas are so powerful when it comes to analysing data, have a look at Chris Adamson’s book “Star Schemas: The Complete Reference”. 

Here at Anatec AI we have many years of experience in modelling data, including business analysis and data preparation for both people and AI. So, if you think your data might not be in the right format to deliver your business goals, get in touch for a chat.

How to prepare data for the Key Influencers visual

AI visuals in Power BI are powerful tools for uncovering hidden patterns — but they depend on having the right data. Power BI’s Key Influencers visual uses machine learning to determine which attributes are most important for a selected outcome.

Unlike traditional visuals such as bar charts or line graphs, where the relationship between data and display is obvious, data for AI visuals require more care and attention. The machine learning model powering Key Influencers can only generate useful insights if your data is clean, well-organized, and rich enough to find patterns.

Let’s look at how to prepare your data — and what to do if the “No influencers found” message appears.

What is the Analyze field in Power BI key influencers?

The “Analyze” field (also known as the Metric) is the target outcome you're investigating. This could be:

• A categorical field (e.g., Green, Amber, Red)
• A binary outcome (True/False)
• A numeric field (e.g., satisfaction scores)

Best practice: Use a field with low cardinality, that is few unique values. This helps the model detect meaningful patterns more reliably.

If your field is continuous like revenue or age, create binned categories. For example, instead of raw scores, use “High”, “Medium”, and “Low” categories.

Also, ensure each row in your dataset represents a unique observation — no duplicates for the same individual, customer, or case.

Choosing Key Influencers “Explain By” fields

The “Explain by” section contains the fields that will be evaluated to understand what might be influencing the thing you want to analyse.

Tips for choosing “explain by” fields:

• Include as many attributes as you have relevant (and clean) data for
• Avoid columns with many nulls or blanks
• Use correct data types (text, numeric, categorical)
• Look for attributes that have sufficient data. More rows give better performance with AI visuals. Aim for at least 100 observations per category — this gives the AI model enough depth to detect reliable patterns.

What to do when Power BI says “No Influencers Found”

Seeing the “No influencers found” error usually means the model couldn't find any statistically significant relationships between your Analyze field and the explain-by variables.

Here’s how to troubleshoot it:

1. Is your data distribution too even?

If your analyze field is too evenly distributed (e.g., 50% Yes / 50% No), there may not be enough variation for the model to detect influence.

2. Clean your data

Missing values reduce the model's ability to draw accurate conclusions. Clean up your data by intelligently filling in blanks or remove incomplete rows.

3. High cardinality in the metric column

If the Analyze field has too many unique values (e.g., raw revenue numbers), you could get the "No Influencers Found" error. Bin or group your metric into logical categories (like “Above Average” / “Below Average”).

Improve AI accuracy with data breadth and depth

When working with AI visuals like Key Influencers, think about both:

• Breadth: the number of attributes (columns) you provide
• Depth: the number of observations (rows)

A wide, clean, and deep dataset gives the machine learning model room to detect complex patterns and relationships — leading to more meaningful and actionable insights.

And if you would like to talk to us about getting your data AI-ready, then get in touch. We love a good data problem! 

How does Key Influencers work?

Microsoft Power BI includes the Key Influencers visualization, which uses machine learning to create insights from your data. The training of the data is done within Power BI, making it quicker and easier to harness the power of AI. There’s no programming or selecting machine learning models – it’s all done for you. That means you have more time thinking about the business opportunity (or problem) and less time worrying about getting it to work. All of which means that Key Influencers is worth another look. 

What problem does Key Influencers solve?

Before we start doing something more efficiently, let’s think why we would use Key Influencers at all. The visualization is designed for businesses who want to track important metrics. There are different names for these metrics, including:

• Key performance indicators

• Performance measures

• Objectives and Key Results (OKR’s)

• Balanced Scorecard measures including financial, customer, internal, and learning and growth

• Scorecards

• Lead and lag indicators

I’m sure there’s more – if you track metrics under a different name, put it in the comments and I’ll add it!

But whatever you call the metric; these are all ways of tracking things or processes that are important to your business. But tracking a metric is only part of the problem. The other half is knowing what to do to improve it, and that can be difficult. Everyone has a view, but it’s expensive to try all possible combinations.

A data-driven approach is more effective. By collecting data on what’s actually happened, you can cut through the decision-making process and get results faster. And that’s what the Key Influencers visual does. You provide it with data that includes actual results for the metric, plus lots of factors that may, or may not, contribute to improving it. Key Influencers uses machine learning to tell you which factors are important.

It doesn’t matter whether you want to influence the metric to be high (like sales) or low (like errors), the Key Influencers visual works in the same way. For example, if your metric is absenteeism, you might have data that includes work patterns, commute distance, job profile, etc. Some of these factors that may be contributing to the problem and Key influencers helps you understand which are important. 

How does Key Influencers work?

The key influencers visual is actually two visuals in one: 

1. Key influencers

2. Top segments 

There is a tab at the top of the visual that allows you to toggle between the two. They both use the same set of data, however, and they both work by using supervised machine learning. This means that the model is trained using data where the outcomes, in this case absenteeism rates, are known. The model then learns the patterns and correlations between the outcome and influencing factors.

• The key influencers visual uses linear regression to show which attributes in your data correlate most strongly with your metric.
• The top segments analysis uses decision trees to segment the data into groups based on the attributes in your data. 

In both cases a sample of the data is used to “train” the machine learning model, before processing all the data and generating the results. All of this goes on behind the scenes, within the Power BI engine. And it does it all impressively quickly. There’s a huge amount of power within this single visual!

What data works best with Key Influencers?

Supervised machine learning works by crunching as much data as you can find. However, it must include actual results relating to whatever you are analysing. For example, if you are analysing absenteeism, you need data for your chosen metric, say number of days absent in a given period (such as month or year). And related to each case of absenteeism, you also need the factors that might affect the metric, such as demographics, job profile, etc.

When data includes known results, it’s called supervised learning. The machine learning algorithms take these past observations and figure out why they happened.

Let’s talk!

Are you using the Key Influencers visual? What do you think? Leave a comment about your experiences – good or not so good! And if you interested in finding out more, let’s talk. 

Unlock the power of AI with Key Influencers

AI is nothing short of revolutionary when it comes to solving tricky problems. Anyone who has been using generative AI such as ChatGPT knows that it’s an order of magnitude more powerful than what went before. But as with any ground-breaking technology, AI takes some getting used to. 

Traditionally, we develop software to carry out tasks based on our own assumptions and knowledge. Now, AI is using data to provide better answers to our problems. It’s still our data, but the power of AI is such that it can make more sense of it than a human can. AI's ability to provide more accurate, data-driven solutions to our problems and opportunities is incredibly powerful.

AI: A new approach 

Let’s say your goal is to increase sales. Traditional marketing would tell you to define your target market using market research and then craft a message tailored to that audience. While this approach works, it’s not always straightforward and by necessity has to include some assumptions. 

AI, on the other hand, uses a different approach: it sifts through large datasets, identifying the segments that are most likely to respond to your message based on past behaviour. And that’s the difference: AI doesn’t rely on what people say they will do; it looks at what they’ve actually done.

It’s not so different to what a market research expert once told me long after he had retired: "Look at what people do, not what they say." That’s precisely what AI does—it analyses historical data to provide insights that are grounded in reality, not assumptions.

Get started with AI with Microsoft Power BI

We tend to think of AI as being expensive and requiring teams of data scientists. In fact, Microsoft Power BI is a great way of getting started. It includes several powerful AI visuals: Key influencers, decomposition tree, Q&A, and Smart Narrative.

The Key Influencers visual uses AI to pinpoint the factors that influence the metric you're tracking. Whether it’s sales volume, net profit, or error rates, the Key Influencers visualization can help you identify what drives your key business outcomes.

What Problems Can Key Influencers Solve?

The Power BI Key Influencers visual works by analysing data to uncover the factors that either increase or decrease your chosen metric. Here's how it works in a nutshell:

1. Identify your key metric – eg sales volume, customer satisfaction, or product defects.
2. Gather relevant data – collect information on various factors that could impact your metric. This could include customer demographics, process efficiency, or external conditions.

Data can come from existing process or created for the project, such as:

• To increase sales – which customer demographics can you identify?
• To reduce errors – which data are you already monitoring? What could usefully be added?
• To increase productivity – which factors might impact employee performance that are already known?

Get your data in shape

For AI to work its magic, your dataset needs to meet certain criteria. Fairly obviously, there has to be some possible cause and effect between your data. There's no value in adding observations that do not influence your metric. In addition:

• Data may be continuous (numeric) or categorical but each row must represent a unique case.
• Data must be clean with no errors, duplicates or outliers that could skew the results.
• Get as much data as you can – AI thrives on large datasets. At an absolute minimum, Microsoft recommends having at least 100 observations for any one factor and at least 10 observations for comparison purposes. But large datasets give better results.

What do the results reveal?

The Key Influencers visual ranks the factors that influence your metric based on their influence, helping you quickly identify the most important drivers. But here’s where it gets really interesting: the AI doesn’t just stop at general insights—it can also segment the data to reveal deeper patterns.

For instance, it might show that "large companies" drive higher sales, which you might already know. But Key Influencers goes further by identifying more specific segments, such as "large companies in France’s textile industry" having a particularly strong impact on sales. These insights are tough to uncover manually, especially when dealing with large datasets.

Next steps: put AI to work

The Key Influencers visualization is just one of many ways that Microsoft Power BI can integrate AI into your business analysis. However, getting your data ready for analysis is crucial for success in any machine learning project.

At Anatec AI, we’ve got a ton of experience in data wrangling and data engineering, ensuring that your data is clean, structured, and ready for powerful AI insights. We also create semantic models and measures, so your data is in the right format for visuals such as Key Influencers. If you want to learn more about how we can help you unlock the full potential of Power BI’s AI features, don’t hesitate to get in touch for a chat.

Data is the new oil

Almost 20 years ago Chris Humby coined the phrase “data is the new oil” and it has been repeated like a mantra ever since. “Data is the new oil” has come to define a generation of business thinking. Data has changed the way we think, and the way we pay for things. 

Chris Humby was a data scientist before we knew that we needed data scientists. He worked with Tesco and found value in a place where no one else had looked: the weekly shop. Back then, each unremarkable purchase of toothpaste or bread was just that - unremarkable. Today we know differently. We now recognise the value in data that shows, for example, the purchase of nappies. We know that it indicates a high probability of household with a baby. Which in turn means a high probability of tired parents likely to redeem coupons against toys or premium brand nappies. And conversely, there’s no point offering money off toys or nappies to households without young children. 

The result was the Tesco Clubcard which became a win-win for both consumers and Tesco. And, of course, Chris Humby didn’t do too badly out of it either. Together, they had struck oil. 

Data needs to be refined

As with many catchy sayings, there’s more than a grain of truth in what he said. Like oil, valuable data is often hidden and needs research to be identified as such. And like crude oil, data also needs to be refined to be made useful. There was investment and experimentation in the Clubcard project before it became the success we know today.

Where is your data hidden?

A lot has happened since then, including of course AI. Whilst AI was around 20 years ago, it was not in the accessible format it is today. Azure AI Services make AI models available on a pay as you go basis, requiring an understanding of software engineering rather than machine learning to implement them.

But a lot of valuable data is still hidden. Some companies have made big investments in data analysis,  but many have not. And almost all businesses have vast amounts of data in databases, spreadsheets, legacy systems, and many other places. This is data that no one has yet identified as valuable.

Competitive advantage in an AI world

In a world where AI makes general knowledge easier to access, competitive advantage will come from business-specific knowledge. That is, the knowledge that individual businesses or organisations have built up over time and is unique to their organisation and industry. This is company-confidential data that has value to decision-makers - provided it can be easily accessed when it’s needed. 

Consider the needs of a medium sized business or department that wants to make in-house data easily available to their people. In the past, this might have been done using a combination of induction training, on-going training courses, operating manuals, and data shared on intranets.

ChatGPT has shown us a friendlier way to access data. AI now makes it much easier to access company data in the same way. Azure AI Search provides functionality to augment large language models like ChatGPT with domain specific data, such as company knowledge bases or information held in a database. 

Azure AI Search

Azure AI Search enables advanced querying of different types of data. It has clever ways of indexing data, which enables better performance when querying the data. And it integrates with other Azure services to provide natural language processing. 

It is no longer news that data has value. Now the question is how business should use their corporate data to best advantage. Azure AI Services, which includes Azure AI Search, puts better data within the reach of more businesses.

If you want to explore what Azure AI services could do you for you, then get in touch. We are Azure software developers with solid experience of software engineering and data manipulation. And we’d love to help you find your oil. 

What can Azure AI Services do for you?

AI is revolutionizing industries worldwide. From automating customer service with ChatGPT to enabling early disease detection, the possibilities are limitless. But how can businesses unlock this power without employing expensive AI? That’s where Microsoft Azure AI services comes in.

Business leaders are rightly focused on results, not building AI models. Azure AI services let you integrate powerful, pre-built AI capabilities into your business applications without requiring a team of specialists or a deep knowledge of machine learning.

Azure AI services offer pre-built, easy-to-deploy AI models that can be provisioned through the Azure portal, just like adding a database or virtual machine. Capabilities like image analysis and document intelligence are now available to anyone who can see an opportunity to improve a business process. Azure AI services make it much easier to turn a good idea into a better business.

Azure AI Services include Language, Speech, Vision, Content Safety, and more. For example, Azure AI Language enables sentiment analysis, key phrase extraction, and summarization—helping you better understand customer feedback and improve decision-making. Some Azure AI services can be customized, allowing you to develop innovative solutions to address specific challenges. You might want to improve customer experiences or streamline operations. And with a pay-as-you-go pricing model, building a proof-of-concept application is low-risk.

Azure also provides a comprehensive suite of tools—such as storage, app services, and containers—that support AI-powered application development. Now, instead of needing AI experts, we can use Azure software engineering skills to create powerful AI solutions.

With Azure AI, businesses of all sizes can tackle challenges and seize opportunities in ways that were once only possible for larger organizations.

If you are ready to explore the potential of AI in your business, contact us now to find out how we can help. We have the Azure software development skills needed to incorporate Azure AI services in ways that could transform your operations and drive innovation.

Retrieval Augmented Generation

I’ve got a riddle for you. When is ChatGPT not the best thing since sliced bread? Answer: When it makes things up! 

What? You didn’t think that was funny? Well, if getting things wrong is a big deal in your work, you can be forgiven for not rolling on the floor laughing.  

There are many industries where it’s important to get things right such as software development, renewable energy, science, medicine, education, government, or any other knowledge-intensive field. In all these areas, people depend on having timely and correct information to do their work and advise their customers.

What are “hallucinations” in LLMs?

Large language models (LLMs), such as ChatGPT and others, have revolutionised the way we interact with information. But despite the impressive communication skills of LLMs, they do have some limitations, such as making things up, also known as hallucinating. It may not happen very often but for some businesses and organisations, it's a big deal.

Hallucinations include generating information that is plain wrong, providing information that’s not relevant, or attributing things incorrectly. But why do they happen? And are the LLMs actually making things up? 

LLM hallucinations often occur because of gaps in the training data. When the question is specialised, relating to a niche area, or outside the time period of data for the training data, the LLM may lack the domain-specific sources and so helpfully does the best it can. 

Sometimes the response is obviously wrong, but often the response is highly plausible. And if the person requesting the information doesn’t have specialised knowledge, they may well think the answer is correct. This is the point where the riddle becomes even less funny than it was to begin with. And it wasn’t very funny even then ….

How Retrieval Augmented Generation (RAG) helps

RAG is the process of optimising large language models (LLMs) for specific knowledge-intensive tasks. The “augmented” in RAG is the process of supplementing the data available to the LLM with additional, domain-specific data. This may be internal to an organisation, or specialist data for a particular sphere of knowledge. It might include searching databases and selected documents. It means that the LLM has access to more recent and more specialist data. Which means that hallucinations become less frequent, and relevance and accuracy are increased for that particular area. And by making verified company-specific data available to the LLM, there’s massive business benefit.

Why RAG matters to business

As our world becomes more complex, so does our need to manage information. In industries like software development, manufacturing, or renewable energy, accurate and reliable information is crucial. By integrating RAG into your business processes, you’re giving your teams better data for decision-making, customer service, and product development. RAG provides a way for organisations to improve their products and services by harnessing AI in a way that’s tailored to their needs. 

Ready to leverage RAG in your business?

The first step in creating a RAG solution is identifying and collecting relevant content ready for indexing. If you’d like to discuss a proof-of-concept project for your business, we’ve got the relevant skills to tailor a solution to fit your specific needs. Get in touch to see whether our skills are a good fit for your goals. 

Is your data AI ready?

Intelligent data: what it is, and why it matters

I know a lot of people are wary of AI. The idea that software might make better decisions than lawyers, scientists, or doctors takes a bit of getting used to. But AI is already improving the world in wonderful ways. 

AI is processing images to improve scientists’ understanding of wildlife populations. Advanced research shows that AI could do a better job than some doctors of identifying eye problems. And AI is already analysing documents to provide lawyers with relevant case histories.

None of this is science fiction. These are real projects that are reducing costs, improving accuracy, and removing repetitive tasks from highly skilled people, freeing them up to do more valuable and interesting  work. 

Whilst we marvel at the intelligence and friendliness of ChatGPT, many businesses executives are trying to figure out how best to use AI to stay competitive.  The irony is that answer may be surprisingly simple.

1. Learn about AI: what it really is, how it works, and why it matters in your industry.
2. Bring data together: AI works best on large volumes of trustworthy data. That means you need to bring disparate data sources together into one unified and secure data platform. 
3. Get your data in shape. Clean it up. Match different data sources. Remove duplicates. Have an audit trail so you can trust the output. Add meta data. 
4. Rinse and repeat. Put processes in place so new data is added and cleaned ready for analysis. 
5. Make it visual. Microsoft Power BI has AI capabilities that make it much easier to visualise data, and to find data insights. 

AI is a disruptor, there can be little doubt about that. But figuring out how it will disrupt specific industries is more difficult to predict. But businesses who use data to uncover insights, and help their people make better decisions are outperforming those who do not. But everyone has to get started by understanding what the opportunities might be, and prototyping the possibilities.

Microsoft and the Azure intelligent data platform is a leader for Analytics and Business Intelligence platforms. From Power BI, Microsoft Fabric, to Azure SQL Database and CosmosDB, there’s a data platform to meet data needs large and small. 

If you are thinking about your next steps into an AI-powered world, get in touch to see whether our data engineering experience could help.

Would your data win a gold medal?

Whether you work for a global enterprise or a local business, data quality is vital for analysis and reporting. I’ve talked about this before in How to Improve Data Quality and Designing for Data Protection but today I’m exploring a structured approach to data quality known as the medallion architecture. 

Data can come from a variety of systems, including legacy systems that have been used for many years. Some well-designed relational database systems will have tight constraints to help improve data quality, but others may have a lot of errors. You therefore need a way of getting your data into a format that can be trusted by decision-makers. 

This architecture comes from the world of big data and data lakes, but the ideas behind it are useful for all data projects. It’s called the medallion architecture, and it processes data in three distinct stages: Bronze, Silver, and Gold—just like Olympic medals. Clever, right?

Bronze layer: raw data

Data from source systems are imported “raw” without making any changes to the data. The purpose of this stage is to:

• Validate import integrity. Ensure no data are missing, the original schema has been preserved, data has not been corrupted, etc.

• Add meta data. Columns are added to identify the import date and time, originating system etc. 

• Provide an audit trail. This bronze layer data is not modified, and so can be used to validate queries that emerge in later stages.

• Avoid re-importing. This initial stage might not be the most glamorous, but it provides the foundation for everything that follows and needs to be done with care. You want to avoid reimporting the data if problems emerge later. 

Data are appended to the bronze layer periodically, and so files will increase in size over time. Data in the bronze layer is never accessed directly by business users, data scientists, or analysts. Instead, it forms the foundation for the silver and gold layers.

Silver layer: clean data

The silver layer uses data from the bronze layer and is never created directly from source data. The purpose of the silver layer is to:

• Clean the data. Fix issues such as missing or null values, deduplicating, dealing with out-of-range values, data types, normalization, and other data quality issues.

• Validate the data. Check no errors have been introduced by comparing and testing against the bronze layer. 

• Normalize the data. Data may be split separate tables reading for processing at the gold layer.

Data is typically not aggregated at the silver layer but if aggregation is done, at least one non-aggregated record is preserved. Data at the silver layer might be used by data scientists or analysts. Business users would normally have access to the gold layer. 

Gold layer: business-ready data 

The gold layer is where data becomes business-reporting-ready. At this stage:

• Data is denormalized and aggregated according to business needs.

• Data models and measures are created, in line with how users want to query and analyse the data.

The gold layer is focussed on optimizing the data for business intelligence reporting. Data presented in a format suitable for business users to work with tools such as Power BI to create dashboards and reports. 

Data-led decisions need gold-standard data 

I may be overdoing the Olympic theme, but the concepts behind the medallion architecture are now considered best practice. It is built on good data practices that actually work. In summary:

1. Separate data ingestion and validation from later stages. Preserve this “raw” version of the data to create a base from which to validate future processing. 

2. Manage data quality issues as an intermediate step before aggregating, modelling or creating measures. 

3. Optimize for business use. Create aggregations and measures suited to business reporting requirements at the final stage. 

If you are tackling a reporting project, moving data to the cloud, or need help improving your data’s reliability, get in touch for a chat. We have decades of experience in improving data quality and data manipulation. Together we could make a winning team to turn raw data into golden insights (alright, alright, enough).

Do you need a data strategy?

Three "A"s to improve your data management

In previous blog posts I’ve discussed the benefits of creating an L&D dashboard and the need to keep historical data, including training data. But a dashboard is only as good as the data behind it. Bad data with visual impact is still bad data. And worse, it will lead to bad decisions without anyone realising why. 

A well-thought-out data management strategy is the necessary step before reporting. It will save you time, money, and will improve decision-making. 

Small and medium sized businesses (SMEs), and departments within large organisations, can all benefit from managing their data in a structured way. A data strategy provides a framework that makes it easier to discuss the costs and benefits of data management. 

In this article, I’m going to discuss how to develop a data strategy to keep your data accurate and trustworthy. 

The basics

There are some basic principles to think about, regardless of what type of business or department you work for. These are the three “A”s of data management:

1. Accurate

2. Available

3. Adheres to the rules

Accurate

Data that doesn’t contain errors sounds simple enough, but mistakes creep into data in all sorts of ways. Incorrect dates, spelling mistakes, the wrong person or the wrong course, etc. Each individual error might not be important, but as you amass more data it all adds up. Whether you are using dashboards to monitor progress, create insights for strategy, or preparing for AI, you need a solid data foundation. 

For data to be accurate it must also be complete. It needs to include all relevant time periods, parts of the organisation, or relevant data. That often means thinking about where data is stored. You are likely to have data in relational databases such as SQL Server, or Oracle, often used as back-end data stores for LMS systems, accounts systems, production systems, etc. Other data will be semi-structured or unstructured, such as data in spreadsheets, Word documents, emails, training evaluation forms, or PDF files. 

Available

Availability means ensuring data is easily accessible to those who need it, when they need it, and in a format they can use. 

1. Granting access to those authorised to use the data and restricting access to those who shouldn’t have access.

2. Providing user friendly data models that can be used to create dashboards or for self-service reporting.

3. Storing data with appropriate redundancy safeguards. Data can be lost either by human error, hardware failures, or other disasters. 

4. Providing training so people know how to access the data, and what they can and cannot do with it.

Adheres to rules

Data rules and regulations come from both inside and outside the organisation. 

Compliance regulations such as GDPR require organisations to safeguard personal information, with heavy penalties for those who do not safeguard sensitive data.

Within your organisation you may have guidelines that define which roles have access to what data to ensure private data stays private. 

Think about who owns the data and who is responsible for safeguarding it. Use metadata (data about data) to identify different types of data within your organisation so you can put policies in place to manage it. 

Creating your data strategy

Get your team together and start the discussion. Identify a project that will provide immediate business benefits, then:

1. Identify your key data sources.

2. Agree who owns and manages each data set.

3. Identify and implement quality control measures.

Data management is the necessary first stage in communicating generating data insights and making data-led decision-making. Azure has a range of data governance, storage, and redundancy options to help you modernize your data management whilst improving accuracy, availability, and adhering to rules. 

If you want to improve your data management let’s talk about how you can get your data in better shape. Get in touch for an initial chat about what you want to achieve. 

6 Reasons you don’t need an L&D dashboard

Creating a learning and development (L&D) dashboard provides many benefits, including getting your data in good shape, getting an overview of L&D activities, and finding insights to help you make better decisions. 

But is it right for you? Here are six reasons you don’t need an L&D dashboard. 

1. You already have an at-a-glance training report (and it's accurate)

If your current system already provides a clear and comprehensive view of completed and planned training, you don’t need an L&D dashboard. But you do need to check that the data is accurate, and up to date. Have employees reviewed their own training data? Can they correct mistakes? Is it complete, or are their gaps in time or for people? Have you verified historical records? 

Data is only valuable if it is correct; otherwise, you risk basing decisions on a shaky foundation. And that’s all the most important if you are considering how to use AI. 

2. Your L&D activities are closely aligned to business goals (and are clear to everyone)

When people understand why the training is important, and what role it plays in achieving organizational goals, you get the greatest bang for your L&D buck. But aligning training with business objectives, and making it visible, isn’t always easy. 

A well-designed L&D dashboard provides this visibility, ensuring stakeholders understand the purpose of what they are doing. If your L&D reporting achieves this level of clarity, you are probably already getting big paybacks for your clear thinking. If not, business goals are a great place to start when thinking about what you want your L&D dashboard to achieve.

3. Your employees are engaged in their learning plans (and can see their training history)

Engaged employees are key to the success of any L&D program. This requires employees having access to their learning plans and a view of what they have already completed. When people understand how their training connects to business strategy, they become more invested in their development and make better decisions.

For organizations with well-designed L&D dashboards, these tools enhance employee engagement. They already know that implementing a dashboard is a game-changer for improving transparency and motivation.

4. Skills gaps are clearly visible (based on today’s and tomorrow’s objectives)

If your systems enable you to identify skills gaps, and prioritize training based on current and future business objectives, you’ve got this covered. 

But for many, a well-designed L&D dashboard can help to identify high-priority areas like onboarding new employees, addressing compliance gaps, or upskilling for emerging technologies. Nothing stands still with L&D.

5. Your L&D data is being linked to employee performance (and metrics are hotly debated)

Organisations that are working on measuring the impact of L&D on employee performance already have a competitive advantage. Metrics such as productivity, quality, and employee retention are being used together with data for training, coaching, and other L&D initiatives. Dashboards help to communicate both data and insights and get everyone thinking about how to improve.

If you’ve started on this journey, you already know the benefits. However, for those ready to explore these connections further, a highly visual dashboard can provide the impact needed to communicate insights effectively to busy stakeholders.

6. Your L&D activities are visible (and everyone is engaged)

Organizations that make L&D activities understandable for all stakeholders tend to achieve better results. Transparency encourages both trust and accountability. 

If your current systems already provide this level of transparency, an L&D dashboard might not be necessary. However, for organizations looking to enhance visibility and collaboration, dashboards can play a vital role.

Conclusion

An L&D dashboard is a valuable tool at every stage of the learning and development journey. With AI helping us make better decisions, accurate and actionable L&D data has never been more important. The key is to evaluate your current capabilities and ensure your data is accurate and up to date.

Whether you need to refine your reporting systems or explore tools like Microsoft Power BI or Microsoft Fabric, the right solution starts with accurate data. Contact us to discover how we can help you build meaningful insights and communicate them effectively.

Should you keep historical training data?

Everything has a price. In a world where people move on and job descriptions evolve, is there really any value in keeping historical training data?

"Plus ça change, plus c'est la même chose." The more things change, the more they stay the same.

While it’s true that storing and analysing data comes at a cost, so does developing and delivering effective training programs. In fact, training programs are significantly more expensive. Not quite as costly, however, as ineffective training—or worse, employees who are not trained at all. That’s probably the most expensive outcome for any business.

The rise of data-led training

Modern data analysis techniques have made data-led education a competitive differentiator. Training is no longer just a “nice-to-have” addition; effective training and workplace learning have become essential for attracting and retaining top talent. Employees who know what they are doing are better equipped to handle customers, deliver competition-beating products, and improve overall operational efficiency.

The value of historical training data

There was a time when historical training data was nothing more than a static snapshot of the past. But today, with the advances in data platform capabilities, training data can be more easily integrated with other data sources, such as HR records, accounting data, and operational data. This enriched view provides actionable insights and helps identify trends that were previously difficult to discern.

Additionally, tools like Microsoft Power BI now include built-in AI capabilities, making it easier to analyse patterns and predict outcomes. The more data you have, the more accurate and reliable your insights become.

AI needs historical data

As we step further into the AI era, retaining historical training data is more important than ever. AI models rely on large datasets for initial training and continuous improvement. Historical data also plays a critical role in benchmarking, allowing organizations to compare the performance of new models against past results to validate improvements.

Keep your data safe 

The Azure data platform offers various solutions for storing data securely and cost-effectively. With options that include data redundancy and “cold” storage tiers, businesses can store infrequently accessed data at a lower cost. Azure also supports structured and unstructured data storage, as well as advanced capabilities like Delta Lake for Microsoft Fabric, which simplifies managing and analysing large datasets.

Make the most of your training data

If your business’s success depends on its education programs, leveraging historical training data is a smart strategy. By combining our expertise in data management with our knowledge of the Microsoft and Azure platforms, we can tailor solutions that align with your specific needs. Reach out to us to explore how you can unlock the full potential of your training data.

Why It Matters

• Improved Training Programs: Historical data helps refine and optimize training approaches over time.

• Employee Performance Tracking: Integrated datasets provide a holistic view of how training impacts employee success.

• Cost Efficiency: Data storage solutions like Azure reduce the cost of retaining valuable data without compromising accessibility.

Ready to future-proof your training programs? Let’s talk.

Measuring training effectiveness

A waste of time and money

In the days when newspapers and magazines were printed on actual paper, advertisers used to complain that half their budget was wasted, but they didn’t know which half. We used to think this very funny - but remember YouTube and cat videos hadn’t yet been invented. It remains a reminder, however, that a lot of time and money can be wasted if you can’t or don’t measure how effective something is. 

But things move on. Paper has gone the way of papyrus, and advertisers can not only measure how effective an advertisement is, but also predict how well a campaign is likely to perform.

In the world of learning and development, things are also changing. Organizations have huge amounts of data that can be used to analyse training effectiveness. The Azure data platform, Microsoft Fabric, and Microsoft Power BI are all accessible and cost-effective ways to manage and analyse data. And AI capabilities help to spot trends that were previously available only to specialists. This means that measuring training effectiveness by matching results to learning activities is now available to all organizations. For companies whose competitive advantage is their people, this is a game-changer.

Where to start? 

Whilst accurate training data is important, it’s crucial to first consider what your people are trying to achieve. Having clear objectives by role provides a roadmap for data collection. 

Some roles are easier to measure than others. Those in sales often have highly measurable objectives including sales closed, or meetings scheduled. In contrast, the output for knowledge workers can be more challenging to quantify. But how can you plan training, or measure its effectiveness, if you don’t know what you are trying to improve?

Data-led training

Modern, data-led organisations no longer see learning and development as different to any other part of the organisation. Data-led training is both possible and valuable. By leveraging tools such as Microsoft Fabric with Power BI, the Azure data platform, together with their AI capabilities, companies can now link learning activities to results. 

With clear role objectives, a well-structured data collection plan, together with learning and development data, the process of measuring training effectiveness starts to take shape. You can then sit back and listen sympathetically as your competitors complain that they don’t know which half of their learning and development budget is wasted. 

We are a Microsoft Partner with a depth of experience in data for learning and development. If you would like to find out more about how Microsoft data technologies can save you time and money, get in touch for an initial chat.