timgrant123.bsky.social
Forensic linguist - academic and practitioner.
Aston Institute for Forensic Linguistics
@aifl.bsky.social
https://www.aston.ac.uk/research/forensic-linguistics
Personal: https://timgrantforensiclinguist.com/home/
68 posts
534 followers
743 following
Regular Contributor
Active Commenter
comment in response to
post
After the release of these three episodes, a new episode will be released the first Friday of each month, starting in April with one of Nicci’s cases.
comment in response to
post
- Episode 3 we discuss a contemporary Scottish terrorism case, and the linguistic work by me and Isobelle Clarke that contributed to the investigation and how we assisted to help resolve the case.
comment in response to
post
- Epidode 2 is an interview with author Kate Summerscale about her fantastic book on John ‘Reg’ Christie, Timothy Evans and the murders at 10 Rillington Place. In this episode we put the linguistic evidence in the case to Kate.
comment in response to
post
So:
- Episode 1 we discuss the confessions ofTimothy Evans to the murder of his wife and baby, his subsequent execution, and then how linguistic evidence was used in the Public Inquiry to cast doubt on the validity of the confession statements.
comment in response to
post
@michaelerard.bsky.social @lynneguist.bsky.social @michaelrosenyes.bsky.social @lingthusiasm.bsky.social
@jwgrieve.bsky.social
@barbarableiman.bsky.social @engmediacentre.bsky.social
@tonymcenery.bsky.social @ans.bsky.social @kirstenp.bsky.social @variola13.bsky.social
comment in response to
post
The other cases in this series are all contemporary, and include the expert witnesses who provided investigative advice or evidence (including Nicci and I).
So subscribe now and then let us know what you think at [email protected]
comment in response to
post
The first two episodes are about the how the 10 Rillington Close murders led to the birth of forensic linguistics, and so we are launching on March 7th - close to the 75th anniversary of the execution of Timothy Evans.
comment in response to
post
There's not many reference lists that include 'Murder on the Orient Express' and 'Silence of the Lambs' alongside briefing notes from The National Crime Agency and the College of Policing, but you get all that here.
Enjoy!
comment in response to
post
A collaboration between forensic linguistics and our colleagues on the literary side of the aisle, led to this look at missing persons appeals both in crime fiction and in reality.
comment in response to
post
Yes - I got an early ‘influencer’ copy (first time I’ve ever been accused of being an influencer!)
But I’d be recommending it anyway
comment in response to
post
This just landed!
Highly recommended
comment in response to
post
Essential reading in forensic linguistics and studies in language and law.
comment in response to
post
Collectively you can hear all three authors’ voices clearly in the text - it’s been updated and reordered, reflecting new research and understandings since the first edition 20 years ago, but it’s a continuation from that earlier text.
comment in response to
post
Just posted a quick response here and on LinkedIn (couldn’t link you there but happy to if you’d like to connect).
Very happy to continue the discussion - genuinely believe it’s important and interesting.
comment in response to
post
All of this said, the primary aim of the paper was to show how we tested LIWC’s performance on a dataset we have interest in.
We hope that it will encourage others who want to use LIWC to first check the precision and recall for themselves on their data and for their questions before using it.
comment in response to
post
Our concern is this overall inaccuracy in both directions will feed into and affect the outcomes of any statistical conclusions from the output, (not to mention the outcome of any summary variables based on the categories).
comment in response to
post
… meaning that the proportions of words represented by the various categories as shown in the main LIWC output are potentially inaccurate.
We also note that precision is very low in a few categories (e.g. ‘aux verbs’ at 49.6%, which could easily be rectified by part of speech tagging).
comment in response to
post
On Ryan’s second point, we understand that LIWC’s design and purpose prioritised precision over recall. However, our findings demonstrate recall rates across multiple categories around or below 70%, …
comment in response to
post
We intended to test on a small sample to see if LIWC could usefully be run unsupervised overall much more data.
The conclusion we drew was that it was not accurate enough (precision and recall) for our purposes.
comment in response to
post
Our motivation was to examine LIWC to see if we wanted to use it in our forensic linguistic research where we examine large data from online deviant communities. Much of our research and some casework is in this ‘big data’ world.
comment in response to
post
Regarding the first point, while LIWC is typically used on much larger datasets than ours and relies on probabilistic language models, we argue that finding the errors at the rate we did in such a small dataset speaks to problems which are very unlikely to improve with more data.
comment in response to
post
… if we can’t disagree well in scientific discussions, there’d be no hope in more emotive domains.
There’s two principal areas we’d like to address from his blog post that we’d like to address explicitly: (1) the size of our dataset and (2) the issue of precision vs recall.
comment in response to
post
Hi Ryan it’s always great to get engagement of any kind on any paper, but in this case particularly from someone as close to LIWC as you are.
I’ve very quickly read through, but will give it more attention later or tomorrow, and get back to you with any thoughts.
comment in response to
post
Thanks Emily
comment in response to
post
This is so neat to see! I was on a (non-forensic) project ages ago where we used LIWC (and now there are companies whose whole product is basically this!) and the results were always suspicious to me. Thank you for doing the work!
comment in response to
post
Thanks for your kind words - we’ve been meaning to get to this for ages. It feels about a decade late!
comment in response to
post
Thank you!
comment in response to
post
Thanks Matthew
comment in response to
post
Yup!
This is one of the many flaws the paper points to… but perhaps not the most surprising !
comment in response to
post
Thanks Daria - we’d be very interested in all/any responses once you’ve had a chance to digest it.
comment in response to
post
Thanks Michael - I’m really interested in responses we receive on this one
comment in response to
post
So as expressed in the opening paragraphs (and evidenced in the paper) our answer to the question...
"Is Linguistic Inquiry and Word Count reliable, efficient, and effective for the analysis of large online datasets in forensic and security contexts?"
...is a resounding "No!"
comment in response to
post
... but linguists have always found it suspect in approach and implementation.
Shockingly, we find no evidence of any independent validation. Our results show that the LIWC coding cannot be relied upon – precision as low as 49.6% and recall as low as 41.7% for some categories.