WELCOME TO EPISODE 17 OF THE TALENTED LEARNING SHOW!
To learn more about this podcast series or to see the full collection of episodes visit The Talented Learning Show main page.
EPISODE 17 – TOPIC SUMMARY AND GUEST:
This is the first time we’ve welcomed a repeat guest to The Talented Learning Show. He’s that good! Today we’re talking about the brave new world of learning analytics with Tamer Ali, Co-Founder and Director of Authentic Learning Labs.
Tamer is a long-time educational technologist and learning systems expert who never stops pushing the envelope in extended enterprise learning and continuing professional development. He has designed, built and operationalized multiple learning software products, and is currently working on a next-generation learning analytics platform.
KEY TAKEAWAYS:
- Effective analytics is more than just data reporting. It involves interpretation and visualization of data in ways that are meaningful and useful for business decision makers.
- Business intelligence tools are not new. However, for multiple reasons, learning organizations and training content publishers have been slow to adopt authentic analytics.
- By leveraging technologies like AI, machine learning and the xAPI standard, innovative analytics tools are making it possible to measure learning in ways that create significant business value.
Q&A HIGHLIGHTS:
How do you define learning analytics?
We think of authentic analytics as a way to help learning and development organizations and training publishers track and analyze learning behaviors in a much more effective, efficient, turnkey way.
What challenges do you see with learning analytics?
When we investigated the market, we found many well-intentioned organizations that are creating really good educational courses and materials. But they haven’t been building a practice of learning measurement in their industry, concentration or field. Even though assessment and measurement of learning impact is a key objective of most initiatives, we saw very little tracking done in a methodical way.
What’s the current state of the learning analytics market, from your perspective?
Many organizations already have solid tools like Tableau and Microsoft Power BI. However, they often lack dedicated resources, or attention, or both. That’s the primary challenge we’re trying to solve.
For years, organizations have used LMS reporting to track things like course registrations and completions. Why should they add outside platforms like yours or Tableau?
REPLAY THE WEBINAR NOW
Yes, reporting is available in many learning systems. Over the last 10 years, it has become popular to pull that data into business intelligence applications like Microsoft Access or SAP Crystal Reports. However, those are fairly low-density, low-impact applications. In other words, the results may be tons of data displayed on maps or charts or spreadsheets. None of it provides quick, useful business insights.
That’s where authentic analytics is different. It’s not just about reporting, but about bringing in meaningful insights to guide decision making. It’s about trying to answer key questions our customers are most likely to ask.
For instance…?
It can be something like, “When is the best time of the year to market and promote courses in this category?” Or, “Which topics are attracting more (or less) interest?”
We’re trying to inform learning and development publishers with data-based intelligence used by other fields like marketing, finance and sales. Learning professionals have equal rights to insights. So we’re providing a specialized toolset. We didn’t invent analytics. However, authentic analytics has been missing from learning and development, so we’re trying to bring that discipline to this space.
What kind of people regularly use your product? What are their responsibilities?
Primarily there are three types of users:
- People who create and oversee learning material on an ongoing basis – This could be the platform administrator or someone who authors content directly. People in this role typically visit multiple times a day because the data is refreshed daily.
- The product owner – Anyone responsible for one or more educational products or training for a line of business. People coming in at that level receive the data either from the dashboards, or from an administrator, or go in themselves. Because of our license flexibility, they’re not restricted. They have full, direct access.
- The overall owner – The stakeholder who needs an executive-level dashboard and/or receives insights from the other two roles.
These people are typically in education or product management functions, but we also see some overlap with IT departments.
Why hasn’t next-level learning analytics been available before?
Well, these tools have been used for a while, but most have some sort of limitation. So, as people mature in their use of these platforms, they realize those limitations and they see a need for a more specialized solution.
For example, tools like Tableau and Power BI have license models that essentially restrict access to an exclusive set of people. Those organizations must provide access at scale, so it can be very expensive for training publishers or associations with multiple product owners and product managers. That approach becomes cost prohibitive.
I see…
Further, for a complete analytics solution, you must invest in hardware to house both the data visualization software and the data warehouse to compile all the source data. This is further complicated by the growing adoption of the xAPI standard for learning activity tracking. Use of xAPI increases the typical data set by multiple folds.
So what extra challenges does that add?
There are multiple related questions:
- Where do we put all this data?
- Who’s going to manage all this data?
- What tools do we actually need?
- Who is able to focus on interpreting this data and build a practice around it?
We’ve seen these issues in organizations of all sizes. That’s why we’ve created a toolset and we provide practitioners to support it. We consider ourselves a kind of BI team in the cloud.
That sounds great – but also complex. Do training teams in associations have the skills for this?
They don’t have the resources. They may have the wherewithal and the capabilities to build these dashboards, but they’re busy developing learning strategy, building new products and analyzing performance. That alone is an all-consuming full-time job. Some may look to their IT team for analytics support, but IT is also stretched with strategic projects. So, all too often, no one addresses the need.
And that’s where you come in?
Right. Our goal is to build a dashboard that is simple and intuitive enough for anybody to adopt very quickly. We want to empower users to develop business insights and make projections within seconds. That’s our core challenge. We take all of the unstructured source data and make sense of it in these dashboards, so you don’t have to do it.
How exactly does that work?
Here’s a recent case study: An organization wanted to justify its impact on assessments. They gave us five years of data – thousands and thousands of rows. If you imported this into an Excel spreadsheet, a typical business intelligence tool wouldn’t do any good. It would take weeks of analysis by experts in statistics, psychometrics and database queries.
But we aggregated those 30,000-40,000 tests attempts in a single visual line graph that interpreted performance on two lines – one represented pre-testing and the other, post-testing. So, in one half-page image, we summarized all that data and put the insights in their hands so they could drill down further. For example, they could uncover which tests work best (or not) within seconds, rather than spending hours and hours on the labor to dig deeper.
Earlier you said xAPI generates a massive amount of data, and that’s helping to drive analytics innovation. For folks who aren’t familiar with xAPI, what is that?
The X stands for “experience.” Really, xAPI is a radical leap forward from SCORM, the de facto learning content standard that ensures interoperability across platforms. xAPI says, “What if learning content includes much richer information?” It captures all of a learner’s activities within a course or a learning experience, and it shares all of that data with learning systems and other learning data record keepers.
So xAPI is a learning standard that doesn’t just focus on whether someone completed a course. Instead, it helps us reveal what they did within that course. xAPI adoption is still in process. It’s not yet widespread, but it’s certainly something that learning professionals should note and consider in their product development plans.
Got it. xAPI provides granular information that hasn’t been available before. But how do organizations make that useful? Do you analyze the data and make it useful for them?
Exactly. If you create custom courses with major authoring tools like Articulate or Captivate, they output xAPI data. We can capture and house that data, and present visualizations that make sense for your business and learning decision makers.
We also offer expertise to say, “Okay, you’ve created a custom course that is producing some very robust data. Let’s create a visualization that interprets that data in a meaningful way and we’ll make it available in your dashboards.”
What about incorporating data from business systems that have nothing to do with learning? Can you combine that operational data with learning data to see a connection with business performance?
Yes. Authentic analytics now extends beyond learning platforms to include complementary tools. So, for example, Google Analytics, Adobe Analytics, Salesforce CRM, and for associations, AMS platforms. Because we have access to that data, we can correlate and connect those data points.
For example, we can see what type of courses attract a particular kind of audience member. Or we can go the other way. For example, if sales performance dips, we can automate content recommendations in the learning platform to bridge the appropriate skill gap.
That’s the kind of powerful intelligence we can provide when we correlate data across platforms. This breaks down system barriers and avoids the isolated “island” nature of learning platforms, and begins to answer valuable business questions.
That makes perfect sense. So, tell me how artificial intelligence and machine learning impact this whole discussion?
Well, it’s not pixie dust anymore. A lot of people talk about these technologies at a high level, but they don’t follow-up with details or examples. That’s why they seem so advanced and out-of-reach to many people. But we use artificial intelligence and machine learning to do the heavy lifting in analytics. In other words, we leverage these technologies to put the burden on machines. And as they get smarter, they understand which feedback is positive and which is negative.
For example, machine learning lets us scale the text responses in evaluations and assessment, so we can score any answer categorically into positive, negative and neutral feedback. So, for organizations that sell or offer learning content to their members or customers, we know what’s favorably received, what’s not, what kind of feedback we’re receiving and what keywords or phrases are emphasized in that feedback.
And so it gets smarter over time. How does it get smarter?
As we capture more data, data informs the machine. And people who run the machine help it refine and polish the way we look at things.
For example, we’ve seen responses like, “No feedback at this point. But if I did give feedback, I would say that the instructor was great.” How do you score that kind of response? It’s a partial positive and a partial neutral.
We continuously learn how to refine things with algorithms that put the power of these machines on our side. The data is essentially the food we need. The bigger the data set, the better the performance of these machines.
FOR COMPLETE QUESTIONS AND ANSWERS, AND FOR ADDITIONAL USE CASE EXAMPLE, LISTEN TO THE FULL PODCAST NOW!
WANT TO LEARN MORE? REPLAY THIS WEBINAR
How to Capture Lifelong Learners: A Holistic Approach to Continuing Education
REPLAY THE WEBINAR NOW
Continuing education can be a lonely experience. Many of us must rely on ourselves to identify credible training sources, choose and consume content, earn certifications and demonstrate our value in the marketplace. But it doesn’t have to be that way.
How can continuing education providers make it easier for professionals to connect with the right resources and navigate through the lifelong learning process?
Find out from our panel of experts:
- John Leh – CEO and Lead Analyst – Talented Learning
- Tamer Ali – SVP Education – Community Brands
- Jacob B. Gold, CAE – Director, Education Development – Community Associations Institute
- Kevin Pierce, MAT – Manager, Digital Learning – American Academy of Dermatology
You’ll discover:
- Why and how to create a lifelong competency model
- How to support self-guided and directed content paths
- How AI helps enhance content recommendations and analyze results
- The value of digital badges and credentialing
- Pricing methods that lock-in long-term subscribers
REPLAY NOW!
Need Proven LMS Selection Guidance?
Looking for a learning platform that truly fits your organization’s needs? We’re here to help! Submit the form below to schedule a free preliminary consultation at your convenience.
The post Podcast 17: Next-Level Learning Analytics – With Guest Expert Tamer Ali appeared first on Talented Learning.
Podcast 17: Next-Level Learning Analytics – With Guest Expert Tamer Ali original post at Talented Learning
No comments:
Post a Comment