Translate

Showing posts with label data socket. Show all posts
Showing posts with label data socket. Show all posts

Sunday

Website scraping with LabVIEW

Scrape or retrieve data from a website using LabVIEW. This is also called data harvesting or data extraction from websites.

One difficulty is finding good HTML data to scrape. Most data like stock quotes have been replaced with dynamic data that doesn't have a clear and consistent text strings that can be searched.

Here is an example that opens a URL website, reads the HTML, picks out a string keyword and displays the data after that keyword. I've chosen a weather site that has a consistent keyword like "temp"> and the page is dedicated to only one city. So the number after "temp"> will be that city's temperature.


This vi will be great for any instrument attached to a network with a web-page to access.

Use the data socket open and then read to capture all the raw text of the webpage.  Then use a string match pattern to search through the text to find the matching strings before the data that you want to scrape.  The example show searches through the text of the webpage to find "temp">.


More details can be found here.