Challenge scraping values from a table
-
Hi,
I have puzzle (or a challenge) that I can't seem to fix...
I want to scrape data from a website.
I am interested in the values of 2 columns (out of 20) and there is a variable number of rows (between 20 and 4000).
The full X-paths of the 2 values of the first row are:
/html/body/form/div[4]/table/tbody/tr/td[1]/table/tbody/tr/td/div/div/div[2]/div[2]/div[6]/div[1]/table/tbody/tr[2]/td[7] /html/body/form/div[4]/table/tbody/tr/td[1]/table/tbody/tr/td/div/div/div[2]/div[2]/div[6]/div[1]/table/tbody/tr[2]/td[11]So
tr[1]is the table header.
Andtd[7]andtd[11]contain the values of interest of each row.I tried a while loop with a counter, but I don't know how to insert the counter value in the tr[counter] line and then use it in the selector.
What would be a good approach for this?
Kind regards,
Rutger van Schoonhoven
-
take a look at this video and see if it helps.
https://www.youtube.com/watch?v=bPC85GSAicQ&ab_channel=AllanZimmermann -
Hey
The prefered way of scraping a table of data is to add a Get Element that gets each row ( TR ) and then add 1 or more Get Elements inside the first Get Element to select the desired element, per row ( TD )
Here is a small example:
https://www.youtube.com/watch?v=rDj2VUjE0so&vq=hd1440 -
@flávio-pinheiro-de-souza This only works for windows components using standard data tables, it will not work for all types of tables, and does not work for other technologies, like Java, IE and NM/chrome, but good thinking
.. I should definitely try and add better scraping options soon. -
@allan-zimmermann
@flávio-pinheiro-de-souza
Thanks for your reply! It looked exactly like what I needed, but I didn't get it to work on my particular table.@Allan-Zimmermann,
Thanks again! It works like a charm.
This is really cool stuff.Kind regards,
Rutger van Schoonhoven
