Does anyone have an starting point for an automated #rstats or #python (because that is what I am most comfortable with) solution for avoiding manually getting the popup tooltip raw numbers in the Tableau NZ respiratory disease dashboard - Virus Detected tab, www.esr.cri.nz/digital-libr... - ThreadSky

thoughtfulnz.bsky.social • 59 days ago

Does anyone have an starting point for an automated #rstats or #python (because that is what I am most comfortable with) solution for avoiding manually getting the popup tooltip raw numbers in the Tableau NZ respiratory disease dashboard - Virus Detected tab, https://www.esr.cri.nz/digital-library/respiratory-illness-dashboard/

Comments

koraunuipakeha.bsky.social•59 days ago

Ten years ago I was the person requesting data. Now where I know my request comes from someone who can manipulate data themselves I give them results in tables as clean and specific as possible to capture the data they need and let them address different questions if they arise

koraunuipakeha.bsky.social•59 days ago

Lovely having the Tableau charts for casual viewers, but csv for the win

thoughtfulnz.bsky.social•59 days ago

I do think both the ESR wastewater and the GNS earthquake data strike good balances for providing (in slightly different locations) data at the level of complexity you want.

chris.bur.gs•58 days ago

any chance of asking ESR to turn on "data" download at https://public.tableau.com/app/profile/esr.nz.surveillance/viz/NewZealandAcuteRespiratoryIllnessIntelligence_esrwebsite/NationalSummary ?

(it looks like Tableau is doing its best to not provide the data in current configuration)

chris.bur.gs•58 days ago

if the data was in the display more accessibly, this would be a fun thing to do ... serving it up as .png feels like it's set to "nope"

thoughtfulnz.bsky.social•58 days ago

I had an informal chat with someone in the preparation chain late last year, and my sense is there is not a lot of resource in the foreseeable future to prioritise nice to have refits of the way the data is provided.

thoughtfulnz.bsky.social•59 days ago

What I can do, to my shame, is (on my Mac) use AppleScript to keep Safari open and active (to show the popups) and shell script calls to move the mouse (Python Quartz.CoreGraphics CGWarpMouseCursorPosition) & screenshot. Then do OCR stuff on the output. But that's horrid, so I'd like to do better.

tslumley.bsky.social•59 days ago

I have no actual experience but someone did point me in the direction of RSelenium, which now suggests this: https://rstudio.github.io/webdriver/

hadley.nz•58 days ago

I’d recommend https://rstudio.github.io/chromote/ these days. And if you need a simple starting point https://rvest.tidyverse.org/reference/read_html_live.html

hadley.nz•58 days ago

I have more of a guide at https://github.com/hadley/web-scraping. You might also try looking for an “unofficial” API

thoughtfulnz.bsky.social•59 days ago

Knowing that PhantomJS exists may be handy in other circumstances, so thanks for that, but the digging my way into the structure of the Tableau visualisation (without a tutorial or similar to start from) is a bit beyond my web skills (unlike looping around mouse position, which I can do Ex nihilo)

merrrrin.bsky.social•59 days ago

Have you tried Selenium or similar?

thoughtfulnz.bsky.social•59 days ago

The main issue I've got there is that getting in to the Tableau visualisation through the DOM model, so being able to manipulate the page to the desired data, is a bit beyond me- unlike iterating the mouse around.
I suspect if I could untangle the graphic I could get the data out more directly

merrrrin.bsky.social•59 days ago

Ah, looking at it now, it’s not a very nice DOM to try to navigate programmatically. I wonder if there’s some keyboard navigation that might feel a bit nicer than hard coded mouse coordinates 🤔

But maybe if it ain’t broke..

Comments

Posting Rules

Reply