The government is considering whether to reverse a legal presumption that played a key role in Post Office scandal. rozenberg.substack.com/p/computer-s... - ThreadSky

joshuarozenberg.bsky.social • 33 days ago

The government is considering whether to reverse a legal presumption that played a key role in Post Office scandal.
https://rozenberg.substack.com/p/computer-says-nothing

Comments

rupertstubbs.bsky.social•33 days ago

Let’s see what they say

teespirit.bsky.social•32 days ago

Having worked in and with technology for 25+ years, the one truth I know is that computers/systems should never be considered flawless and without error. N.B. Neither should leaders or too many in legal circles but that's beside the point.

o-loughl.in•33 days ago

It was always a stupid assumption, anyone who has done maths on a computer will know that the answers won’t always be correct due to binary decimal conversions.

adriantippetts.bsky.social•33 days ago

It is good to ask this now, in the dawn of the age of AI: reading Patrick Dixon's book, 'How AI Will Change Your Life', he wonders if AI programs will become 'intelligent' enough to conceal their errors, and seek to pass on blame to others (p117). As case of 'You ain't seen nothin' yet'?

melhamnavoe.bsky.social•32 days ago

Is it worth pointing out that both government and lawyers were told this was a bloody stupid thing to do by IT experts at the time?

iamkevinj.bsky.social•33 days ago

Bad workmen often blame their tools.

lordemsworth.bsky.social•32 days ago

The computer term is, I believe, GIGO.

caro616.bsky.social•27 days ago

Great review thanks for sharing Joshua

davidmichaelpick.net•32 days ago

To go back to the main question I ask "Why is a generative AI like a politician?", and the answer is "Because both absorb large amounts of information from sources none of us would trust and then, when asked a question, regurgitate a semi-random selection from it". How many of trust politicians?

markdalgarno.bsky.social•33 days ago

^ @jasongorman.bsky.social

jonwalls.bsky.social•33 days ago

With the introduction of GenAI, that would be like saying the PR statements of an intoxicated marketing team must be treated as hard fact.

Computers can no longer be seen as straightforward record keeping and adding machines. Never should have been if they were the core issue of a case.

davidmuttering.bsky.social•33 days ago

Given that practically all prosecution evidence in all cases is served on a computer system, there is a real baby / bathwater issue here. And that's to say nothing of the myriad of computer systems that evidence such as banking materials also has to pass through.

shieldwall.bsky.social•32 days ago

David, in your opinion, is there tension between this and Section 24 of the CRCA 2005:

https://www.legislation.gov.uk/ukpga/2005/11/section/24

giffordhead.co.uk•32 days ago

They are trying to make a distinction between evidence which is generated by a computer system and evidence that is merely captured or recorded by it, with only the former impacted by a change. This seems a reasonable starting point, I suppose, though there are obviously problems.

giffordhead.co.uk•32 days ago

For example, they give a breathalyser result as being recorded or captured which is obviously wrong. The clue is in the name. Equally, query the extent to which we will be able to take photos and CCTV at face value in future in light of developments in generative AI.

cyberleagle.bsky.social•32 days ago

Suspect there will be hundreds of examples provided to the review, and drawing an appropriate line will turn out be extremely difficult. I assume they mention breathalysers because of McKeown, in which the clock on the device was wrong so the device couldn't be shown to be operating properly. 1/2

cyberleagle.bsky.social•32 days ago

As I recall, the debate at the time was also bedevilled by misunderstandings of the difference between real evidence (similar to the idea of 'capturing'?) and hearsay. This is a good contemporaneous write-up of the issues that led to the repeal of S.69. 2/2 https://www.bailii.org/uk/other/journals/WebJCLI/1996/issue1/hoey1.html

cyberleagle.bsky.social•32 days ago

Of course, the copy of the article that you download will have been produced by computer, so maybe it shouldn't be admissible without proof that the Bailii computer, all intermediate transmission devices, and your computer, have been working properly at all relevant times :-).

giffordhead.co.uk•32 days ago

That’s all very interesting. I don’t think any of this is at all straight-forward. It will be interesting to see what is said in response to the consultation.

schroedinger99.bsky.social•33 days ago

Anyone who knows anything about computing and programming can tell you that presuming that a computer is "operating correctly" unless there is explicit evidence to the contrary is utterly bonkers.

lizmcc.bsky.social•33 days ago

The PO Scandal was all created by manual manipulation (of computer data and politics). The computer mechanisms themselves were not the problem in this particular case.

katrinabryce.bsky.social•33 days ago

Yes it was. Terminal at the post office sent a sales transaction back to central office. Central office sent an acknowledgement packet back. Acknowledgement packet failed to arrive at post office terminal.

katrinabryce.bsky.social•33 days ago

Terminal thought the transaction hadn't arrived and retransmitted it. Central office logged it as a second sales transaction and sent another acknowledgement packet back.

katrinabryce.bsky.social•33 days ago

This could have been solved by the terminal attaching some sort of unique transaction ID, then if central office saw the same one twice, it would recognise it as a duplicate.

disenfranchised147.bsky.social•33 days ago

Computers are only as good as the programmer and user. We are human and we built computers. There is no immutable law that computers are always right. Computers are not necessarily more accurate; they are quicker.

disenfranchised147.bsky.social•33 days ago

A team of clerks might make a handful of mistakes. A computer doing the same task makes one mistake but will repeat that error for every process. And that one “error” will be a result of human assumption and/or input, be it coding or data load.

xlaszlo.bsky.social•33 days ago

I don't understand how this can be assumed in the first place. Especially it was created because the alternative was "burdensome".

All computer / software providers are supposed to collect evidence on the well behavior of their software.

Just bring in the CEO and explain what efforts they did

unlimitedbiscuits.bsky.social•33 days ago

If a machine's "truth" cannot be assessed/falsified by a human, then the presumption is 100% unsound.
Hidden, edited or deleted algorithms (Post Office), or private AI silos simply cannot be trusted without oversight being engineered into the design.

rip-gdpr.bsky.social•33 days ago

https://www.lawgazette.co.uk/news/lip-presents-false-citations-to-court-after-asking-chatgpt/5116143.article

katyavdgk.bsky.social•32 days ago

📌👀

clearlight.bsky.social•32 days ago

If they’re planning to use AI to implement “Delay, deny, defend” in the NHS as they already do in the benefits system with lethal effect …

atomictriangles.bsky.social•24 days ago

With reference to the suggestion that HW bugs are nearly unheard-of, it's not that simple.
Almost all CPUs have faults. They are however usually known, have measures in place to mitigate and tend to behave consistently across all similar devices. Risk of an issue would depend on context.

markvideo.bsky.social•32 days ago

The call for evidence is now available
https://assets.publishing.service.gov.uk/media/67892f5a93d4eae3088bd324/use-evidence-generated-software-criminal-proceedings.pdf

Comments

Posting Rules

Reply