What is Machine Learning and How does it Work?

What is Machine Learning?

Machine learning is a fancy term for ‘data-driven analysis’ frequently applied to making decisions. Essentially, there’s some set of rules we have settled on in the past and are now applying classified data in a new way that predicts an outcome with more accuracy than just throwing dice or flipping coins would have afforded. Machine learning can be cool stuff! You can practice and solve some awesome machine learning projects at ProjectPro. For example, classifying 500K Tweets into hashtags can yield 1 million insights about the world.

Machine Learning vs. Deep Learning vs. Neural Networks

There are three primary mechanisms out to classify a data set from this insane volume of information that can generally be broken down into algorithms across the ages. The first, traditional classification is based on the assumption which is fine but doesn’t tell us much about how many positive or negative correlations there may exist between two variables when one variable only accounts for 33% (.33) of total variability…or roughly how “sparse” our training sample must become in order for a computer to be able to classify them above chance.

The second, Deep Learning and Neural Networks assume that the data is sparse…generally referring to under-represented (for some random mathematical reason) instances of the variables amongst their training set with this assumption allowing incredibly fast classification times while accounting back…in theory…to how likely it would have been that stochastic variance in one variable isn’t causing fluctuations in another even though it’s perfectly plausible that it could.

Machine learning algorithms in modern times either have to make do with this assumption or deploy the third solution using both algorithmic and practical machine learning techniques as fit coins, Gaussian mixture models, non-negative matrix factorization (NNMF), etc. How well the training sample spreads out depends largely on how tedious you find creating your data set for science even though I’d advise not arbitrarily throwing in rotten fruit and mushrooms to ruin the test.

Multi-class problems such as what gets classified are usually helped by having correlated variables present, so let’s take a look and see how much we can glean from comparing these three very different classifications: Traditional Classification – Decided upon how many categories their training sample is divided into (extremely sparse) When there aren’t that many really distinct possible combinations it generally appears as dot or bar plots but if you divide them up into small enough dimensions they look like these random lines and bars Traditional Classification – Decided upon how many categories their training sample is divided into (extremely sparse).

When there aren’t that few really distinct possible combinations it generally appears as dot or bar plots but if you divide them up into large enough dimensions they look more like this Random looking line data set Yup, no correlation in the choice between two classes 0.1% of the time for each classification. That’s an indicator that neither classification particularly detects the true distribution if there is one for sure, and it appears as though we can make further progress by adjusting our data set to coincide with this assumption: How about having some correlation in determining when a new sample gets classified When you really do have information on what kinds of pixel values mean say by using regression models or cluster analysis.

How machine learning works – A practical introduction to machine learning

Machine Learning is the area of AI that most people would be interested in, so let’s forget about regression (which doesn’t really help us) and instead look at more directly defining features we’ll be looking for. Traditional Classification: – Decided upon how many categories their training sample is divided into … But there are an awful lot of possible samples to classify as well How about using just a few dimensions for our samples, those that define the “type” of the pixel we want to classify?

One example might be if you are trying to determine whether a scanned image has been turned sideways or forwards. You get 25 samples in total and there are displayed with their X-axis on top, can’t just throw them at some regression function without knowing more about why we think these pixels look like they do first! So instead I’ll try an automatic distinction between different categories.

Machine learning methods

Since the invention of the computer, business and science have had an increased demand for pattern recognition and algorithmic decision-making that has led to computational methods becoming standard.

In areas such as extracting concrete meaning from natural language, designing more efficient infrastructure, improving predictions in poll surveys, rendering video without human intervention, and making unmanned systems reliably autonomous (SOA), machine learning can offer vastly technical advances.

As a means of prediction, machine learning techniques offer powerful new methods.

The academic study and application of technology are known as Artificial Intelligence (AI). The field draws heavily on computer science in general and statistics in particular – many actual researchers have master’s or doctorate degrees but those titles are not always required to practice the profession today. Scholars interested solely in artificial intelligence often call the subfield Machine Learning while some practitioners use other terms like algorithmics or data.

Supervised machine learning

In supervised machine learning, algorithms are trained using pairs of data from a smaller set (known as the “training set”), and a standard test score that uses this subset is then fed into the analysis. The algorithm learns which features correspond best with its intended purpose or prediction. If you have 5 days worth of weather measurements, for example – how good will your model be at predicting what it’ll be on the 100th day?

In cases where we do not have a training set, like the output of an algorithm we want to see better performance on test data. The most common cause is known as scenarios:

If no standard has been established for predicting these outcomes (for example in a predictive model that is trying to sell you insurance, disaster management, or any other real-world application), one first needs to build a “standard” or baseline by testing each outcome under free choice.

Unsupervised machine learning

In unsupervised learning the algorithm is given no intended purpose: it simply examines an arbitrarily large data set. The vast majority of work on AI nowadays uses this approach, but its historical use in medical diagnosis and computer vision was limited (as computers do not have a visual language developed to describe features). The goal for supervised learning is to match these features to the label “red.”

These goals are typically expressed by a machine learning engineer as policies based on specific desired outcomes. Unsupervised models can be fed with thousands of hypothetical questions and determine their semantic meaning.

Semi-supervised learning

In semi-supervised learning, the algorithm is given one goal and asked to score against a set of samples for which it has not been trained. For example, imagine we have an offense vs defense soccer simulation game using teams from different decades; the underlying data contains the team’s playing styles at different points in history (with historical player statistics).

Each outcome will be scored by computing how well each team did over time – that scoring could then be used as input for a model that learns which team’s have historically had a good offense, defense, and special teams. Such models may only be trained on slightly larger or fewer data sets than supervised machines.

(note: Some semi-supervised algorithms will require you to endow the algorithm with prior knowledge of your labels, such as “we want this outcome”) Semi-supervised learning can also sometimes suffer from overfitting/confusion in prespecified environments if the algorithm is given incomplete data sets – conditional on the consistent outcomes there are complex optimal learning strategies (e.g., in the Netflix recommendation system).

Reinforcement machine learning

In the context of tying signal to action, reinforcement learning (RL) models learn task-prescribed value functions. RL techniques commonly have features that make imperfect analogs of AI cognitive architecture such as recursion or hierarchical representation mapping. Commonly used RLS include Q-learning and A* search among others.

The goal is to approximate experience maps so well that meaning can be inferred with collected data without any knowledge-base; this is a robust process that learns to maximize the chance of a given valued outcome. Mentally, it seems like an instantiation of Lamarckism – since those intelligent enough to learn how to capture value generally learn and pass on successful knowledge via behavior (over generations) until their genes adapt.

Reinforcement learning models were originally developed in game theory as hoped-for solutions for reliable interaction between opposing players with imperfect information about each other’s moves and are constant time.

Real-world machine learning use cases

In science and engineering, there are many real-world use cases for machine learning that make the intersection of artificial intelligence with data sets too rich to be addressed in one article.

Samples range from analyzing molecular interactions at a finer scale than previously possible due to deep-learning models, identifying relationships within gene regulatory networks or protein structures; functional genomics such as predicting human diseases through machine learning inference on synthetic biology modules; putting together drug repurposing datasets from former drugs from “clinical trial dump” to large datasets; detecting nutrients and elements in biological samples, identifying weak correlations between genomic variants associated with specific health outcomes.

About learning, machines learn abstract concepts, such as generative models or associational hierarchies which cannot be explicitly defined by the programmer. In these cases, performance is accomplished through training data sets statistically conditioned on unseen examples of system behavior rather than hand-editing code or feeding parameters into a simulator.

For example, learning to identify meaningful patterns in data sets is called unsupervised learning while improving algorithm performance by training on both desired associations and as-yet-undiscovered ones are supervised learning. Predicting the behavior of genes or protein structures is part of known unknowns (KU) problems, where it may involve interaction mechanisms between different parts necessary for robust adaptivity under severe perturbation.

Machine Learning in Speech Recognition:

The goal of Machine Learning techniques is to learn from unlabelled data in an effort to generalize or classify more accurately and quickly than a purely human analyst. The discriminative power of machine learning algorithms thus far has largely been demonstrated by speech recognition, especially when speakers were made as well-known as celebrities such as Morgan Freeman (with feature engineering) and Xiahouwei Hu (without), who became the first humans shown on CNN’s morning

Machine Learning in Customer Service:

Training a computer system to provide better customer service requires an accurate, machine-readable model of customers.

There are several contexts where such models can be used:

Digital signatures & Passwords: The ability to authenticate electronic documents using cryptographic techniques is enabling new market opportunities for companies as consumers demand more secure authentication security solutions and require the use of passwords that cannot be compromised by hackers or stolen from passwords management databases.

Neiman Marcus then, utilizing Deep Learning to authenticate credit card transactions, which require more security than a traditional password. Increasing popularity in this market segment has prompted competitors such as Intel and Microsoft to build their own facial recognition technology for recognizing the shopper at her checkout counter entry point.

Enlarging social media audiences: Companies that allow consumers to spend money on a marketing campaign or brand exposure benefit from leveraging online data from various sources including direct input from customers into personal websites (Web Riders), applications (Facebook accounts, Pinterest accounts), or social networking sites such as Twitter.

This audience demographic can reportedly be as large as 1 billion users (as of 2012). These volumes together make up a potentially valuable marketing inventory that has yet to saturate and will only increase so long as personal data becomes more widespread online

Personalized opinion ranking: While consumers on Facebook do not want their friends list permanently rearranged by predictive algorithms saving useful features from us and foregoing friends-based algorithms. Marketers may still want to use Facebook’s “like” functionality for simple crowdsourcing when a brand reaches out with quantities of small messages or basic offers (promotions, discounts..etc) for example on BuzzFeed through specific hashtags related to the subject matter at hand.

Machine Learning in Computer Vision:

Ford fitted more than 8,000 vehicles with its new self-driving technology that is capable of detecting obstacles and indicating their safe driving path

Companies using Computer vision are benefiting from the ability to train machines on what a real human would have actually seen. This represents tremendous potential for users to lazy load messages without waiting in order to enable faster response times when needed or simply improve efficiency by automatically identifying images or applications used across various online destinations.

Machine Learning in Recommendation Engines:

Exploring domain-specific applications in recommending content such as music and movies by harvesting an enormous number of user comments, faves, ratings, etc.

Google’s intent is to eventually turn its search results into a kind of digital recommendation engine that “learns” from millions upon millions of users’ feedback (online discussions are considered predictive data). Ideally, each query will result on the most relevant websites or organized collection thereof across different categories along with the expected click.

Machine Learning in Automated Stock Trading:

Intuition, algorithms, and a shrewd investment manager can make for an effective combination that many interesting has achieved success in the financial industry

E*Trade was acquired by TD Ameritrade among others. Bear Stearns was bought out by JP Morgan Chase resulting from its trading robot which provided “massive” cuts to operating margins at scale while automating much of the process. hedge funds come & go but the aforementioned tech wins remain intact over time when maintaining a balance between reliability and speed with data & analytics.

Machine Learning in Social Networking:

Facebook Newsfeed, Grouper groups on LinkedIn, Hashtags on Twitter, etc (I couldn’t find any data to back this up though) Web 3.0 future – Pandora lends its platform for developing highly personalized user profiles by listening to music taste instead of merely sorting through a million tracks from the usual percentage of the world’s population which could decrease marginally.

Machine Learning in Programming:

Serverless computing  – where functions used to be hosted somewhere on the cloud but now they can reside in local RAM and scheduled via code. This abstracts away infrastructure management which allows for a huge brain-like Google or IBM to handle machine learning without being limited by administrator privileges, system memory limitations, or other tricky problems like exposing some storages of your computer (Linux/etc.)

Machine Learning’s Contributions:

Machine learning algorithms have helped in all areas of programming, including neural networks for classification as well as reinforcement-learning methods for robotics and machine translators. Machine translation is notoriously difficult; human expert systems can provide only successfully translated 500 bits per second, while the maximum rate achieved by a commercial CNN – which interprets outlines from 100 to 1 billion words at once using backpropagation  – is 20 words or so per minute.

Challenges of machine learning

Are we stuck with machine learning?  Though the number of ML algorithms may go down as other rapid advances in AI/ML get released. There is little doubt for most programmers here: The problem stems from fickle models – given a chance to know what will happen, they use their knowledge and explicitly specify the environment or data-set that embraces it. We need more conservative estimates on these rates or else many programs would fail since they are strongly uninformed.

AI impact on jobs

The artificial intelligence revolution poses a number of challenges. One key question is how the shift away from traditional computers will affect job prospects and economic growth, especially for low-skilled workers whose main function has long been to operate machines.

Most people believe that AI will lead to new types of employment opportunities rather than eliminate existing jobs as some predict or, at best presage mass unemployment.

Privacy in Machine Learning

Intelligence is obviously tied to privacy. However, up until now knowledge and understanding of what constitutes information needed for data mining have been rather vague; for example, there was little idea about who expected or would benefit from such a service. Suppose that the public was informed by academia, research partners, and government agencies but not by corporate organizations – how do you think it might affect their operations?

This stems from companies’ secrecy in sharing information with universities, research partners, and governments which they usually do not publicize.

The Technological Singularity: Implications of Machine Intelligence B3WIRE indicated that after the singularity all living humans will be greatly enhanced and an extraterrestrial intelligence (SETI) might have already discovered us – if we did discover it first then after this discovery there could be a different situation.

Bias and discrimination by Machine Learning

‘Bias and Discrimination in Machine Learning’ has shown the existence of statistical biases that can unnecessarily decrease or increase certain outcomes. For example, if we were to ask a machine learning system to learn from data without knowing precisely how it was generated but with incorrect assumptions regarding the process, this could result in inaccurate results due to systemic bias which also leads us to discrimination issues since these may cause some forms of prejudice toward any identifiable group.

Data and Blockchain Technology 

The technical aspect of data is the process involved with its collection, analysis, storage, securing from exposure to destruction or theft. However, there are a few issues that need to be addressed such as how we can ensure confidentiality when blockchain records are immutable; whereas regular files should have timestamps showing they were viewed at some point in time and potential bias in the data due to natural occurrence and personal preferences.

The need for privacy is considerable so that a person may be able to inspect, update or add their own information which will decrease record tampering or self-delusion by an identity thief when they are trying to believe they’ve stolen someone else’s information.

Machine Learning Accountability

The machine learning research community has taken an interest in accountability by considering the potential for bias and discrimination as a problem when dealing with large data sets from multiple sources.

FAQs

Conclusion

In this article, we talked about what machine learning is and how it works. We broke down the topic into the three different types of machine learning: supervised, unsupervised, and reinforcement. We also went over how they relate to artificial intelligence and the concept of deep learning. Finally, we covered some of the everyday uses of machine learning, including the online services that are built on top of it.