Screen Scraping

What is Screen Scraping?

screen scraping is the process of extracting data from a graphical user interface (GUI) using a computer program. This can be done by identifying and capturing the elements of the GUI that contain the desired data, such as text, images, and buttons.

Screen scraping can be used for a variety of purposes, such as:

• Collecting data from legacy applications. Some legacy applications do not have APIs that can be used to extract data. Screen scraping can be used to extract data from these applications by capturing the data that is displayed on the screen.
• Completing tasks that are difficult or impossible to do manually. For example, screen scraping can be used to automate the process of filling out forms or submitting data to websites.
• Creating new applications or services. Screen scraping can be used to extract data from existing applications and websites to create new applications or services.

How Does it Work?

Screen scraping works by first identifying the elements of the GUI that contain the desired data. This can be done by using a variety of techniques, such as:

• Inspecting the HTML source code of the GUI. The HTML source code will contain information about the structure of the GUI and the elements that it contains.
• Using a screen scraping library. Screen scraping libraries provide tools that can be used to identify and capture elements of a GUI.

Once the desired elements of the GUI have been identified, they can be captured and extracted. This can be done by using a variety of techniques, such as:

• Taking screenshots of the GUI elements. The screenshots can then be processed to extract the desired data.
• Using a screen scraping library to capture the data directly from the GUI. Screen scraping libraries can typically capture text, images, and buttons from a GUI.

The extracted data can then be stored in a database or used in a new application or service.

Screen scraping is a powerful tool that can be used to extract data from a variety of sources. However, it is important to be aware of the legal and ethical implications of screen scraping. It is important to obtain permission from the owners of the GUI before scraping their data. Additionally, it is important to be careful not to overload the GUI with requests, as this can cause performance problems.

Is screen scraping legal?

Under PSD2 — the EU law designed to increase competition in the payments industry — screen scraping is still legal as long as certain security steps are followed, including identification of the TPP to the bank it is accessing. However, most banks have moved to providing APIs to enable access to account data and payments, so screen scraping is not necessary.

There has been plenty of debate on whether screen scraping should be banned entirely. In the UK, most banks provide APIs for access, and some banks that have supported screen scraping have now been required to transition to APIs. In Europe, the European Banking Authority (EBA) has advocated to end the practice, but industry consultations are still ongoing.

What’s the difference between open banking and screen scraping?

Open banking is a way of giving regulated companies secure, limited access to your bank account, with that customer’s permission. Previously, that information would only have been accessible by banks. Open banking has led to several examples of new and innovative services that help consumers and businesses make the most of their finances. Open banking can also include payment initiation, where TPPs make payments on behalf of their customers with their consent.

Screen scraping is effectively one way to power open banking. While other technologies are becoming more and more common, screen scraping is still accepted under PSD2 when more modern and secure API technology isn’t available or working.

More reference:

https://truelayer.com/blog/product/what-is-screen-scraping/

https://research.aimultiple.com/web-scraping-vs-screen-scraping/