Just published something about how we’re designing GOV.UK Chat, a product that’s currently in a private beta across some GOV.UK pages insidegovuk.blog.gov.uk/2024/11/28/h... - ThreadSky

kubabartwicki.bsky.social • 97 days ago

Just published something about how we’re designing GOV.UK Chat, a product that’s currently in a private beta across some GOV.UK pages

https://insidegovuk.blog.gov.uk/2024/11/28/how-were-designing-gov-uk-chat/

Comments

kubabartwicki.bsky.social•97 days ago

Our users already use tools like this, so I’m glad that we’re thinking about what it means for us by making and experimenting

kubabartwicki.bsky.social•97 days ago

Personally, I think that a conversational experience done well* might be able to help people get things done a lot more easily, and could link you to real humans when you need them, but let’s see what people think of this thing first

(* this bit is really hard!)

robinpem.bsky.social•97 days ago

Thanks Kuba - this is really helpful - shared it with the students this week as example of user centred product design with AI... (they're building their own MVP ideas this term) - some of the other AI inspiration they're looking at here - https://kingstonuniversity.padlet.org/rpembrooke/ai-inspiration-this-week-creative-entrepreneurship-cu9s7jldtytsr8iu

tomski.com•97 days ago

Thanks for the excellent transparency. Much respect for this.

Interesting that nearly 1 in 5 people using GOV.UK Chat do not realise that it can give inaccurate answers.

Any sense of whether this 20% is likely to be biased towards those most vulnerable?

kubabartwicki.bsky.social•94 days ago

Great points, thanks Tom.. Nothing in our data suggesting that those users are more vulnerable so far. It’s not that 1 in 5 didn’t realise, but that 4 in 5 did – subtle but important difference – most of that 20% were unsure so would hopefully still check answers

tomski.com•94 days ago

I just don't think hoping is enough. It's one thing for a company to have an LLM chatbot pointing people towards wrong answers. It's another for a government to do so.

tomski.com•94 days ago

That said, everything's a trade off - maybe share what you know about the trade offs?

tomski.com•97 days ago

Would be interesting to estimate the likely cost of failure of GOV.UK Chat. (And compare with existing cost of failure for GOV.UK website.

https://tomloosemorework.wordpress.com/2024/04/18/ai-and-the-cost-of-failure/

l8nm.bsky.social•97 days ago

I like these questions at the end, and would add 'level' to frequency, impact and cost. How wrong could the tool be? What happens if there's an absence of info for the tool to find? Lots of internal testing to do

dracos.co.uk•97 days ago

My credit card’s fraud department asked me to get in touch today on the website today, and they did this by printing “Click Chat bottom right and type ‘CCP’ to skip the bot and go straight to a person.”(!)

peterbjordan.bsky.social•97 days ago

1/ I'm interested in GOV.UK's choice to use Generative AI to produce 'new' info as opposed to using AI to parse the query and point to the existing carefully-designed and approved content.
And - how does the gen AI content stand up against GOV.UK's styleguide.

peterbjordan.bsky.social•97 days ago

2/ Have you done any research into how chat is presented within GOV.UKm ie in the context of the whole site - what choices do users make? Is there any confusion?

kubabartwicki.bsky.social•94 days ago

We explored and tested a bunch of options for how we describe it, where we put it, and how we integrate it.. There is definitely more work and iteration to do there

Comments

Posting Rules

Reply