Data Scraping

What is data scraping?

Data scraping, in its most general form, refers to a technique in which a computer program extracts data from output generated from another program. Data scraping is commonly manifest in web scraping, the process of using an application to extract valuable information from a website.

https://www.cloudflare.com/en-in/learning/bots/what-is-data-scraping/

How Data Scraping Is Done 

 Web scraping is a fairly direct process when viewed at a high level. Code is utilized to pull information, generally via a scraper bot. The bot sends a request to the website, parses the HTML document and converts it into a different format.

Over time, the game has grown more sophisticated. As scraper bots become successful, content protection strategies are beefed up to thwart their efforts. In turn, the bots respond by developing tactics to outmaneuver these new protection mechanisms — and so it goes.   

For the scrapers, content may be derived at little or no expense. Instead of having to write their own content, conduct research and obtain customer reviews, for example, the scrapers may post material on their sites. They avoid having to pay for certain reports and other documents.  

Two Types of Data Scraping

Web Scraping

If you’ve ever copy and pasted information from a website, you’ve performed the same function as any web scraper, only on a microscopic, manual scale.

Web scraping, also known as web data extraction, is the process of retrieving or “scraping” data from a website. Unlike the mundane, mind-numbing process of manually extracting data, web scraping uses intelligent automation to retrieve hundreds, millions, or even billions of data points from the internet’s seemingly endless frontier.

Screen Scraping

Screen scraping is the act of copying information that shows on a digital display so it can be used for another purpose. Visual data can be collected as raw text from on-screen elements such as a text or images that appear on the desktop, in an application or on a website. Screen scraping can be performed automatically with a scraping program or manually with an individual extracting data.

How is web scraping stopped completely?

The only way to totally stop web scraping is to avoid putting content on a website entirely. However, using an advanced bot management solution can help websites eliminate access for scraper bots almost completely.

What is the difference between data scraping and data crawling?

Crawling refers to the process large search engines like Google undertake when they send their robot crawlers, such as Googlebot, out into the network to index Internet content. Scraping, on the other hand, is typically structured specifically to extract data from a particular website.

Sources / References:

https://www.datamation.com/big-data/data-scraping/

https://www.cloudflare.com/en-in/learning/bots/what-is-data-scraping/

Cryptocurrency

 Abstract on Cryptocurrency:

Cryptocurrencies have emerged as important financial software systems. They rely on a secure distributed ledger data structure; mining is an integral part of such systems. Mining adds records of past transactions to the distributed ledger known as Blockchain, allowing users to reach secure, robust consensus for each transaction. Mining also introduces wealth in the form of new units of currency. 

Cryptocurrencies lack a central authority to mediate transactions because they were designed as peer-to-peer systems. They rely on miners to validate transactions. Cryptocurrencies require strong, secure mining algorithms. In this paper we survey and compare and contrast current mining techniques as used by major Cryptocurrencies.

 We evaluate the strengths, weaknesses, and possible threats to each mining strategy. Overall, a perspective on how Cryptocurrencies mine, where they have comparable performance and assurance, and where they have unique threats and strengths are outlined.

Facebook Digital Currency - Diem (Libra)

Facebook Digital Currency - Diem ( known as Libra). Offering single-currency stablecoins in addition to the multi-currency coin

The Diem Association’s mission is to enable a simple global payment system and financial infrastructure that empowers billions of people. The Association’s first step toward creating a more inclusive and innovative financial system began in June 2019 with the announcement of the project. Our goal was to establish a collaborative dialogue early in the journey.

 We have worked with regulators, central bankers, elected officials, and various stakeholders around the world to determine the best way to marry blockchain technology with accepted regulatory frameworks. Our objective is for the Libra payment system to integrate smoothly with local monetary and macroprudential policies and complement existing currencies by enabling new functionality, drastically reducing costs, and fostering financial inclusion.

Blockchains have a number of unique properties that can potentially address some of the problems of accessibility and trustworthiness. These include distributed governance, which ensures that no single entity controls the network; open access, which allows anybody with an internet connection to participate; and security through cryptography, which protects the integrity of funds. 

But existing blockchain systems have yet to reach mainstream adoption. Mass-market usage of existing blockchain-based currencies has been hindered by their volatility and lack of scalability, which have, so far, made them poor mediums of exchange.

What Is Diem?

Diem is a stablecoin similar to Tether (USDT) and other price-pegged cryptocurrencies. It runs on the Diem project’s own Diem blockchain, and the coin is held in a wallet, called Novi.

Naturally, Diem will be used for payments. However, the Diem blockchain will also be programmable, much like Ethereum, so that developers can create custom apps. Diem uses a custom smart contract language called Move that uses a logical constraint called “static typing” to guarantee its security.

The Diem blockchain is powered by validator nodes, which are run by members of the Diem Association. These nodes confirm transactions and validate blocks. This diagram outlines the process:

Image Source: https://cryptobriefing.com/diem-introduction-facebooks-stablecoin/

Efficiency is an important part of the Diem blockchain. Diem’s testnet has been criticized for handling between 6 and 24 transactions per second, far less than non-blockchain payment networks like Visa and Mastercard. But in practice, Diem’s blockchain may be faster, as it aims to support 1000 transactions per second.

On top of this, Diem is promising near-zero transaction fees. On Diem, interest is paid to validators that comes from reserve assets. That means Diem does not need to take a 1%-3% cut of each transaction to make money, as payment companies currently do.

History of Diem

Diem has been under development for nearly two years in various forms. These are the most important milestones in its history.

Early History

In May 2018, Facebook messaging Vice President David Marcus was appointed to a new blockchain division within the company. Marcus previously was president of PayPal and a director on the board of Coinbase. These facts led many commentators to predict that Facebook was planning a crypto-based payments platform.

Further clues emerged as Facebook hired more crypto developers, who eventually leaked details to the press. The Wall Street Journal was one of the first to report on the company’s plans.

At that time, Facebook’s crypto product was supposedly intended for use via WhatsApp in India as a remittances tool. That plan has apparently been abandoned in favor of broader goals.

Official Announcement

In June 2019, the Facebook-backed Libra Association officially announced Libra and revealed the structure of the project in detail.

The Libra Association announced that the blockchain would be powered by a consortium of 27 major companies operating independently of Facebook. Meanwhile, Facebook would profit from those plans by building the Calibra wallet for the cryptocurrency (now called Novi) and by having a single governance vote.

Though the Association remains active, not all of its original members remain involved. By the end of 2019, several members of the Libra Association left the project, including Visa, Mastercard, PayPal, eBay, and Stripe, as well as Mercado Pago and Booking Holdings. That forced the project to rebuild its Libra Association with the remaining members and new additions.

The project also faced numerous regulatory roadblocks at the time of its announcement, including pushback from the U.S. Senate and several European regulators. Libra’s 2020 launch date came and went, and the project remained mostly silent throughout 2020.

The Diem Rebrand

In December 2020, Libra returned with a new name, Diem, and announced plans to go live in early 2021.

What’s the Difference Between Diem and Bitcoin?

If you were introduced to cryptocurrencies through Bitcoin like most people, then forget everything you know. The Diem Dollar is very different from Bitcoin, both in the way that it works and the way it’s controlled. 

For starters, the Diem Dollar is a stable coin built on a permissioned blockchain network. Unlike Bitcoin, it’s not designed as an open network which anyone can participate in. Another difference is that Diem is directly connected to the US Dollar. This distinguishes it from Bitcoin, which isn’t connected to any specific stable asset making its value volatile and unstable. This means that a Bitcoin’s value changes a lot more unpredictably, whereas Diem’s value won’t fluctuate any more or less than the price of a US Dollar.

Another crucial difference between the two is that Diem is a centralized currency. It’s centralized by the Diem Association which oversees its development and the ways in which it can be used. That’s worlds apart from Bitcoin, a decentralized cryptocurrency that isn’t overseen by any one person, group or company. 

Finally, since the Diem Dollar is pegged to a fiat currency, it isn’t connected to any finite resource. The US Dollar is a currency whose value is in direct correlation to the number of Dollars in circulation and is no longer connected to the amount of gold that the US owns. This makes the number of Dollars that can be printed virtually unlimited. By contrast, the number of Bitcoin in circulation is very much finite, with a cap of 21 million in existence.

By introducing Diem, Facebook and the Diem Association are changing the way we imagine cryptocurrencies. Unlike Bitcoin, Diem isn't introducing a new method of digital payment. It’s simply a new way of exchanging money digitally. This should make it a lot easier for non-tech savvy individuals to start using cryptocurrencies to make digital payments for everyday things like groceries and taxi rides.

Conclusion 

Cryptocurrencies have been around for a long time, but the Diem Association could be the group that makes them available to billions globally. By creating a stable coin backed by the US Dollar, the Diem Association has created a new type of cryptocurrency –– one that’s accessible, easy to understand, and easy to use in everyday life. Combine that with Uber, PayU, and Shopify’s user base, and Diem is well on its way to becoming the dominant cryptocurrency over the next few years.

Sources / References Content From:

https://cryptobriefing.com/diem-introduction-facebooks-stablecoin/

https://www.diem.com/en-us/white-paper/

https://www.theguardian.com/technology/2019/jun/18/what-is-libra-facebook-new-cryptocurrency

https://en.wikipedia.org/wiki/Diem_(digital_currency)

https://www.firstdag.com/blog/why-diem-is-better-than-bitcoin/

Quantum machine learning

 Quantum machine learning Abstract:

Quantum Machine Learning bridges the gap between abstract developments in quantum computing and the applied research on machine learning. Paring down the complexity of the disciplines involved, it focuses on providing a synthesis that explains the most important machine learning algorithms in a quantum framework. Theoretical advances in quantum computing are hard to follow for computer scientists, and sometimes even for researchers involved in the field. 

The lack of a step-by-step guide hampers the broader understanding of this emergent interdisciplinary body of research. Quantum Machine Learning sets the scene for a deeper understanding of the subject for readers of different backgrounds.

The author has carefully constructed a clear comparison of classical learning algorithms and their quantum counterparts, thus making differences in computational complexity and learning performance apparent. This book synthesizes of a broad array of research into a manageable and concise presentation, with practical examples and applications. 

 

What is Quantum Machine Learning?

Quantum machine learning is a research area that explores the interplay of ideas from quantum computing and machine learning.

For example, we might want to find out whether quantum computers can speed up the time it takes to train or evaluate a machine learning model. On the other hand, we can leverage techniques from machine learning to help us uncover quantum error-correcting codes, estimate the properties of quantum systems, or develop new quantum algorithms.

Quantum machine learning (QML) is built on two concepts: quantum data and hybrid quantum-classical models.

Why Quantum Machine Learning?

In 2017, Microsoft CEO Satya Nadella explained the difference in computing power and method of classic and quantum computers using an example of a corn maze. The modern-day classical computers would use the brute force and backtracking algorithm to find a path through the maze.

It would choose a path, hit an obstruction, backtrack to the original starting point, choose another path and continue until it finds a way out. It will surely find a solution but at the cost of a lot of time. Imagine your mobile is draining its battery and the algorithm is running for long with no final solution.

This is where quantum computers come to rescue. They unlock amazing parallelism and traverse every path in the corn maze simultaneously to find you an optimal solution in very less time and an exponentially reduced number of steps. It’s like sending a ‘n’ number of drones to the ‘n’ number of paths and get all the results, i.e. path information in unit time.

Where Can We Apply Quantum Machine Learning?

• Model classical data on quantum computers, or create novel quantum- inspired classical algorithms for faster computation and better results.
• As the feature space of the problem domain expands, the computations become really expensive for classical computers. Using superposition and other quantum properties, quantum machine learning helps extensively in kernel evaluation and optimization.
• Quantum machine learning also has the capability of mapping the trillions of neurons in our brain and decoding the genetic makeup.
• Supervised learning and adaptive layer-wise learning with the help of quantum classifiers and neural networks

More seminar topics related to Quantum Machine Learning:

Quantum Cryptography

Quantum Internet

Quantum Processing Units

Quantum Supremacy

Quantum Network

Quantum Logic Gate

Quantum neural networks

Sources / References:

https://www.researchgate.net/publication/264825604_Quantum_Machine_Learning_What_Quantum_Computing_Means_to_Data_Mining

https://pennylane.ai/qml/whatisqml.html

https://www.geeksforgeeks.org/working-of-quantum-machine-learning/

https://en.wikipedia.org/wiki/Quantum_machine_learning

Brain computer interface

Brain computer interface Abstract:

A brain-computer interface (BCI), also referred to as a mind-machine interface (MMI) or a brain-machine interface (BMI), provides a non-muscular channel of communication between the human brain and a computer system. With the advancements in low-cost electronics and computer interface equipment, as well as the need to serve people suffering from disabilities of neuromuscular disorders, a new field of research has emerged by understanding different functions of the brain. The electroencephalogram (EEG) is an electrical activity generated by brain structures and recorded from the scalp surface through electrodes. Researchers primarily rely on EEG to characterise the brain activity, because it can be recorded non-invasively by using portable equipment. The EEG or the brain activity can be used in real time to control external devices via a complete BCI system. A typical BCI scheme generally consists of a data acquisition system, pre-processing of the acquired signals, feature extraction process, classification of the features, post-processing of the classifier output, and finally the control interface and device controller. The post-processed output signals are translated into appropriate commands so as to control output devices, with several applications such as robotic arms, video games, wheelchair etc.

Brain-Computer Interfacing: An Introduction

The idea of interfacing minds with machines has long captured the human imagination. Recent advances in neuroscience and engineering are making this idea a reality, opening the door to restoring and potentially augmenting human physical and mental capabilities. Medical applications such as cochlear implants for the deaf and deep brain stimulation for Parkinson’s disease are becoming increasingly commonplace. Brain-computer interfaces (BCIs) (also known as brain-machine interfaces or BMIs) are now being explored in applications as diverse as security, lie detection, alertness monitoring, telepresence, gaming, education, art, and human augmentation. This introduction to the field is designed as a textbook for upper-level undergraduate and first-year graduate courses in neural engineering or brain-computer interfacing for students from a wide range of disciplines. It can also be used for self-study and as a reference by neuroscientists, computer scientists, engineers, and medical practitioners.

Reference Link:

https://ieeexplore.ieee.org/document/7449615

http://bci.cs.washington.edu/

https://en.wikipedia.org/wiki/Brain%E2%80%93computer_interface

https://www.atera.com/blog/cognitive-systems-teaching-technology-to-learn/

Digital twin (DT)

 Abstract on Digital twin (DT):

Digital twin (DT) is one of the most promising enabling technologies for realizing smart manufacturing and Industry 4.0. DTs are characterized by the seamless integration between the cyber and physical spaces. The importance of DTs is increasingly recognized by both academia and industry. It has been almost 15 years since the concept of the DT was initially proposed. To date, many DT applications have been successfully implemented in different industries, including product design, production, prognostics and health management, and some other fields. However, at present, no paper has focused on the review of DT applications in industry. In an effort to understand the development and application of DTs in industry, this paper thoroughly reviews the state-of-the-art of the DT research concerning the key components of DTs, the current development of DTs, and the major DT applications in industry. This paper also outlines the current challenges and some possible directions for future work.

What is a digital twin?

A digital twin is a digital representation of a physical object or system. The technology behind digital twins has expanded to include large items such as buildings, factories and even cities, and some have said people and processes can have digital twins, expanding the concept even further. The idea first arose at NASA: full-scale mockups of early space capsules, used on the ground to mirror and diagnose problems in orbit, eventually gave way to fully digital simulations.

But the term really took off after Gartner named digital twins as one of its top 10 strategic technology trends for 2017 saying that within three to five years, “billions of things will be represented by digital twins, a dynamic software model of a physical thing or system".  A year later, Gartner once again named digital twins as a top trend, saying that “with an estimated 21 billion connected sensors and endpoints by 2020, digital twins will exist for billions of things in the near future."

In essence, a digital twin is a computer program that takes real-world data about a physical object or system as inputs and produces as outputs predications or simulations of how that physical object or system will be affected by those inputs.

Why and How to Design Digital Twins?

As mentioned above, digital twins can be created for a wide range of applications, for example, to test a prototype or design, assess how a product or process will work under different conditions, and determine and monitor lifecycles.

A digital twin design is made by gathering data and creating computational models to test it. This can include an interface between the digital model and an actual physical object to send and receive feedback and data in real time.

Data

A digital twin requires data about an object or process in order for a virtual model to be created that can represent the behaviours or states of the real world item or procedure. This data may relate to the lifecycle of a product and include design specifications, production processes or engineering information. It can also include production information including equipment, materials, parts, methods and quality control. Data can also be related to operation, such as real-time feedback, historical analysis and maintenance records. Other data used in digital twin design can include business data or end-of-life procedures.

Modelling

Once the data has been gathered it can be used to create computational analytical models to show operating effects, predict states such as fatigue, and determine behaviours. These models can prescribe actions based on engineering simulations, physics, chemistry, statistics, machine learning, artificial intelligence, business logic or objectives. These models can be displayed via 3D representations and augmented reality modelling in order to aid human understanding of the findings.

Linking

The findings from digital twins can be linked to create an overview, such as by taking the findings of equipment twins and putting them into a production line twin, which can then inform a factory-scale digital twin. By using linked digital twins in this way it is possible to enable smart industrial applications for real world operational developments and improvements.

Where is it Used?

Digital twins are used in a wide variety of industries for a range of applications and purposes. Some notable examples include:

Manufacture

Automotive

Retail

Healthcare

Disaster Management

Smart Cities

References: 

https://ieeexplore.ieee.org/document/8477101

https://www.networkworld.com/article/3280225/what-is-digital-twin-technology-and-why-it-matters.html

https://en.wikipedia.org/wiki/Digital_twin

https://www.twi-global.com/technical-knowledge/faqs/what-is-digital-twin

Microsoft Hololens

 Abstract

Seminar on Hololens is Microsoft’s take on augmented reality, which they call “mixed reality”. Using multiple sensors, advanced optics, and holographic processing that melds seamlessly with its environment, These holograms can be used to display information, blend with the real world, or even simulate a virtual world. 

Microsoft HoloLens, known under development as Project Baraboo, are a pair of mixed reality smartglasses developed and manufactured by Microsoft. HoloLens was the first head-mounted display running the Windows Mixed Reality platform under the Windows 10 computer operating system. The tracking technology used in HoloLens can trace its lineage to Kinect, an add-on for Microsoft's Xbox game console that was introduced in 2010

What Is HoloLens?

HoloLens is an untethered, fully self-contained Windows 10 computer that rests comfortably on your head. It’s what’s known as a mixed reality device, a device that tries to blend the real and digital worlds. You see objects placed in the world that look and—to an extent—act like they’re in the real world. In contrast, VR immerses you in an environment and you typically don’t see anything around you but that virtual world. You generally aren’t visually aware of the real world outside your head-mounted display (HMD).  This experience can take you flying in outer space while you sit in your office chair. And AR tries to enhance the world around you with extra data, such as markers, or heads-up information that may pertain to your location. Some AR headsets simply throw text and images on a screen overlapping whatever you’re looking at.

With the HoloLens, you can bring applications and objects into the world around you that understand your environment. If you want an application pinned to the wall or in mid-air like a digital screen,  no problem. Such apps stay put, even when you leave your room and come back the next day. I’m constantly leaving virtual windows open in other rooms, to be surprised when I go back days later and they’re still there. And that’s not all. Suppose you want a skeleton standing in front of you in your living room that you can walk around and inspect (including climbing on your couch to look at the top of the head). Again, no problem. Drop a virtual 3D object, say a ball—referred to as a hologram—into your world and it will fall and hit your real table and stop. Move the table and the ball will fall and hit your real floor. The HoloLens understands the world around you and most are absolutely amazed the first time they try it (though I’m still waiting to be able to download Kung Fu into my brain).

How does it work?

The Hololens has a plethora of optical sensors, with two on each side for peripheral “environment understanding” sensing, a main downward facing depth camera to pick up hand motions, and specialized speakers that simulate sound from anywhere in the room. The Hololens also has several microphones, an HD camera, an ambient light sensor, and Microsoft’s custom “Holographic Processing Unit” that they claim has more processing power than the average laptop. All this comes together to sense the spatial orientation of the unit in the room, track walls and objects in the room, and blend holograms into the environment.

Reference:

https://www.gvsu.edu/cms4/asset/7E70FBB5-0BBC-EF4C-A56CBB9121AECA7F/7_things_about_microsoft_hololens.pdf

https://en.wikipedia.org/wiki/Microsoft_HoloLens

https://docs.microsoft.com/en-us/archive/msdn-magazine/2016/november/hololens-introduction-to-the-hololens

Digital Currency

Digital currencies, also known as e-money, are now more popular than ever. Until recently, digital currency lacked the security to eliminate the need for paper money. However, with the rise of the internet and the advent of blockchain technology, this is no longer the case. Today, digital money is set to change the market forever.

Importantly, digital currencies fill all the uses of traditional forms of money. You can purchase goods or pay for services via these technologically superior financial alternatives. Digital money provides users with instantaneous transactions and more transparency in the market. As such, more countries plan to unveil some form of digital currency in the next couple of years.

What Is Digital Currency?

Digital currency is a form of currency that is available only in digital or electronic form, and not in physical form. It is also called digital money, electronic money, electronic currency, or cyber cash.

History of Digital Currencies

An American computer scientist by the name of David Chaum is credited with developing the first concept for digital currencies way back in 1983. By 2990, Chaum created a working model of his theory dubbed – DigiCash. The concept was years ahead of its time. Consequently, it never gained the momentum needed to survive in the market.

The first recorded public use of digital currency in a wide-scale emerged in 1996. The currency, known as e-gold secured millions of active users before it was shut down by government officials in 2008. From that point, a myriad of corporate-sponsored digital currencies entered the market.

All of these digital currencies encountered a problem known as “double-spend.” Basically, developers struggled to develop ways to ensure that each digital currency could only be spent one time during transactions. This issue saw resolution with the introduction of the world’s first Cryptocurrency – Bitcoin.

How does the Digital Currency Work?

There are many different types of Digital Currency that could be found all around the world. The working of all these different currencies are basically almost the same. The most difficult part to understand in the working of digital currency is the blockchain network. The blockchain is a technology that is the foundation for all the digital currencies, but they are not bound to any single type of currency. Blockchain of digital money is a specific protocol that the digital currency follows. This protocol helps the creation and use of digital money. Some criteria are specified by the blockchain protocol that must be unique to every currency being produced. These criterions are as follows:

• Unique and non-replicable
• Limited in supply
• Durable
• Divisible
• Uniform
• Non- refundable.

Blockchain protocol is compulsory while developing a digital currency. Once the digital currency is created, it must be accessible to people for use. Here comes the concept of “mining”.

The digital currency’s network is a place where all sorts of transactions related to the currency have taken place. Any new transactions are divided into “blocks”. These blocks are confirmed by the miners and validated by multiple users throughout the network before they are added to the end of the chain. Once the block is confirmed, it is sealed and recorded on the public blockchain ledger.

After the digital currency is mined, or transferred between users, it must be stored in some place. That storage space is called the Digital currency Wallet. These are software that is capable to keep the digital currencies securely for an infinite period. The digital currencies are encrypted. So, they have a public key and at least one private key. Some digital wallets are secured using the feature of multi-signature.

The Future of Digital Currency

Given the current state of the world, digital currencies are set to explode in the coming year. Today, there is a robust digital infrastructure in place to support the mass adoption of these currencies. Additionally, their development is being catapulted into the spotlight thanks to the Coronavirus pandemic. You can expect to see this trend continue as more people across the world gain access to high-speed internet. For now, digital currencies such as Bitcoin continue to reshape the international communities’ definition of money.

References

https://en.wikipedia.org/wiki/Digital_currency

https://www.investopedia.com/terms/d/digital-currency.asp

https://www.securities.io/what-is-a-digital-currency/

https://techlabuzz.com/what-is-digital-currency-how-digital-currency-works/

Facebook Digital Currency - Diem (Libra)