ChatGPT is pretty amazing. If you haven't tried it yet, we'd urge you to give it a go. This AI platform is a bit like using Google, except you can talk to it in real, human sentences. And rather than just giving you a string of search results to wade through, it'll then answer your question directly and comprehensively, in plain English.At the same time, though, even the makers of ChatGPT admit that it's still very much a work in progress. In practice it can often be vague, miss the entire point of your question or, worse still, give you information that's completely incorrect. Even when it does give an answer that sounds true, it's very difficult to check its authenticity because it rarely provides a source for the supposed facts it shares. ChatGPT is, though, constantly being updated and improved. So we thought we'd check how it's getting on, by testing it out on a subject we know a fair amount about: employee benefits insurance. Did it give us useful answers or just send us down a rabbit hole? Read on to find out. Question 1: picking a providerWe figured we'd get straight to the point, by asking ChatGPT the following question…"I work for a company of around 100 people, with branches in several parts of the UK and Ireland. What insurance company can offer me the best employee benefits?"It responded: "Selecting the best insurance company for employee benefits for your company of 100 people in different parts of the UK and Ireland will depend on several factors, including your specific needs, budget, and the preferences of your employees. It's advisable to work with an insurance broker or consultant who can help you navigate the market and find the most suitable options. However, here are some well-known insurance providers that offer employee benefits packages in the UK and Ireland to consider."ChatGPT then went on to list 10 companies, but seemed to have picked them purely on the basis of how well-known they were; it certainly didn't offer any evidence that they were the best, or even explain how each company's approach to insurance differed from one another. So ultimately, we didn't really get any more useful information than if we'd Googled "Top 10 insurance providers in the UK". (Trying this, we got a similar, if not identical list).Question 2: Focus on employee wellbeingMaybe, we thought, we should take a different tack and try to narrow things down for the AI. So we asked ChatGPT a follow-up question. "I am particularly interested in improving the physical and mental health of my employees. Is there a group insurance plan that can help with that?"Its answer was, again, a little vague and unfocused. Here's the first paragraph:"Yes, many group insurance plans offer options and features that can help improve the physical and mental health of your employees. These plans often include wellness programs, mental health support, and preventive care services. Here are some aspects to consider when looking for a group insurance plan that focuses on employee health and wellbeing…"ChatGPT then shared advice on 12 specific things to look for. The first was "Wellness Programmes", and the specific advice was "Look for insurance plans that include wellness programs aimed at promoting healthy lifestyles. These programmes can offer incentives for employees to engage in activities such as exercise, nutrition, and smoking cessation." Fair enough, but again we felt this answer was a bit obvious and lacking detail. The other 11 pointers, just for the record were: Mental Health Coverage, Employee Assistance Programs (EAP), Telemedicine Services, Preventive Care, Fitness and Wellness Discounts, Health and Lifestyle Coaching, Mental Health Resources and Apps, Employee Feedback, Network of Healthcare Providers, Educational Resources and Employee Engagement. Each of these came with a sentence explaining what they were, but with no specific information on individual companies or what benefits they had to offer in these areas. Question 3: Narrow things downIf at first you don't succeed, try, try again. So for our third question, we tried to back ChatGPT into a corner and get it to provide some specific recommendations. We drilled down into the first part of its previous answer, and asked: "What group insurance provider has the best Wellness Programme?"Again, a little frustratingly, we got what – if a human had said it – might be considered waffle. "Determining the 'best' group insurance provider with a wellness programme can vary greatly depending on your organization's specific needs and priorities," ChatGPT responded. "Different providers offer different features and levels of customisation in their wellness programmes. To find the best fit for your organization, consider the following aspects when evaluating group insurance providers and their wellness programmes…"It went on, but you get the idea. ChatGPT was either refusing or unable to give specific information on specific companies or their policies. Question 4: Same question, different wordsEver noticed how, when a politician avoids a question on TV, the interviewer then asks the same question in another way? Well, we decided to try that with ChatGPT. So our next question was simply: "What group insurance provider has the best Wellness Programme in the UK?"Those extra three words shouldn't really have made a difference: we'd already told ChatGPT we were at a UK-based company. But funnily enough, it made all the difference… and we finally got to the heart of the matter. "As of my last knowledge update in September 2021, specific rankings or assessments of group insurance providers with the 'best' wellness programmes can vary and may change over time," it told us. "The perception of what constitutes the 'best' wellness programme can also be subjective and depend on the unique needs and preferences of your organisation."So there we have it. ChatGPT knows it's working off old data, and is unwilling to give details of current company policies or practices for that reason. Unfortunately, that means that for finding relevant information on individual companies' group insurance policies, it's basically useless. But what about more generic questions on group insurance policies: how would it fare there?Question 5: Going genericWe asked ChatGPT a series of more general questions on group insurance and, in all honesty, it came up trumps. For example, when we asked: "What is a waiver of premium?"It responded: "A waiver of premium is a provision commonly found in insurance policies, including life insurance and disability insurance, that offers financial protection to policyholders in the event they become unable to pay their insurance premiums due to disability or serious illness."That's pretty much correct, even if it's a bit dry and technical in the telling. We got similarly accurate-if-dull answers to questions like "What type of people would need a certificate of insurance?", "Why might a pre-existing condition impact the cost of a health insurance policy?" and "What are the different ways to pay for employee benefits insurance?"The less generic and more specific things got, though, the more ChatGPT struggled. For example, we asked…"Have any UK laws been passed in the 21st century that affect employee benefits insurance?"… and the list it delivered featured two amendments to UK laws that we could find no reference to anywhere in reality. This is presumably an example of ChatGPT's famed "hallucinations" in which it provides information that appears well-founded but turns out to be factually incorrect or simply nonsensical. (Ironically, you can get a great explanation of this phenomenon by asking ChatGPT: "What are ChatGPT hallucinations and why do they happen?".)Perhaps just as importantly, it failed to mention the Health and Care Act 2022, which has a number of provisions in this area. Although, this is not surprising because, as we've established, ChatGPT isn't aware of anything that's happened since 2021.ChatGPT is a powerful AI system, but one that's still under development. That means it can be a useful tool for finding general information on a variety of topics, but it's not always reliable when it comes to providing specific or specialised information.In the context of employee benefits insurance, ChatGPT is particularly limited. It can provide general information about different types of insurance plans and what to look for when choosing a provider, but it's not able to recommend specific companies or policies. This is because ChatGPT is not actually connected to the internet, and so not able to keep up with the latest changes in the market.If you're looking for help choosing a group insurance plan for your company, then, it's best to consult with an insurance broker or consultant (which, to be fair, ChatGPT recommended we did from the start). They can help you assess your specific needs and budget, and they can recommend providers and policies that are a good fit for your company.You can also reach out directly to our team for advice, whether it’s choosing the right employee benefits package for your organisation or simply to find out more about the different types of group insurance available.About YuLife.YuLife is working to reimagine the insurance industry by protecting lives, rewarding living and inspiring life. We’re on a mission to transform traditional insurance into a life-enhancing experience each employee will value and use daily. How does it work? Our award-winning app uses behavioural science and game mechanics to reward your people for living well while offering protection in case of crisis. And with our top-rated employee assistance programme, your team gets access to mental, financial and social support, virtual GPs, nutritionists, life coaches and more to help them live their best lives.Because we believe that your employees should benefit from their insurance from day one – and that wellbeing should be accessible every day, for everyone.Request a demo for your team today.