GPT-3, a machine learning model from OpenAI, has taken the world by storm over the last couple of weeks. Natural language generation, a branch of computer science focused on creating texts from a batch of data, entered a golden age with last years release of GPT-2. The release of GPT-3 last month only confirmed this. In this article, I want to take a look at why GPT-3 is such a huge deal for the machine learning community, for entrepreneurs, and for anyone working with technology.
GPT-3 is a 175 billion parameters Transformer deep learning model. That might sound complicated, but it boils down to an algorithm that was taught to predict the next word based on the sentence you input. After you provide a sentence and the algorithm fills in the gaps. For example, you could put in How to successfully use content marketing? and you would get a text on the subject of content marketing.
GPT stands for Generative Pre-Training. The generative part of that term should be clear. You want the model to generate a text for you based on some input. Pre-Training refers to the fact that the model was trained on a massive corpus of text and its knowledge of language comes from the examples it has seen before. It doesnt copy fragments of texts verbatim, however. The process involves randomness due to the fact that the model tries to predict the next word based on what came before, and this prediction has a statistical component to it. All this also means that GPT-3 doesnt truly understand the language its processing; it cant make logical inferences like a human can, for instance.
GPT-3 doesnt feature a real breakthrough on the algorithmic side. Its more of the same as GPT-2, although it was trained with substantially more data and more computing power. OpenAI used a C4 (Common Crawl) dataset from Google, which Google used in training their T5 model.
So why is GPT-3 amazing at all? Its transformative nature all boils down to its applications, which is where we can really measure its robustness.
Imagine you want to build a model for translation from English to French. Youd take a pre-trained language model (say BERT) and then feed an English word or sentence into it as date along with a paired translation. GPT-3 can perform this task and many others without any additional learning, whereas youd need to fine-tune earlier machine learning models like BERT on each task. You simply provide a prompt (asking sentence or phrase):
Translate English to French: cheese =>to getfromage
Providing a command without additional training is what we call zero-shot learning. You gave no prior examples of what you wanted the algorithm to achieve, yet it understood that you wanted to make a translation. You could, for example, give Summarize as an input and provide a text that you wanted a synopsis of. GPT-3 will understand that you want a summary of the text without any additional fine-tuning or more data.
In general, GPT-3 is a few-shot learner, which means that you simply need to describe to it a couple of examples of what you want, and then it can figure out the rest. The most surprising applications of this include various human-to-machine interfaces, where you write in simple English and get a code in HTML, SQL, Python, or app design in Figma.
For example, this GPT-3 powered app lets you write How many users have signed up since the start of 2020? The app would then give you an SQL code: SELECT count(id) FROM users WHERE created_at > 2020-01-01 that does just that. In other words, GPT-3 allows you to make queries about spreadsheets using natural language English in this case.
Another great GPT-3 powered app lets you describe a design you want in simple English (Make a yellow Registration button) and get Figma files with the button ready to be implemented in your app or website.
There are plenty of other examples that feature GPT-3 translating from English to a coding language, making the interaction between humans and machines much easier and faster. And thats why GPT-3 is truly groundbreaking. It points us towards new, different human-machine interfaces.
So what does GPT-3 offer entrepreneurs, developers, and all the rest of us? Simplicity and the increasing democratization of technology.
GPT-3 and similar generative models wont replace developers or designers soon, but they will allow for wider access to technology, be that designing new apps, websites, or researching and writing about various topics. Non-technical people wont have to rely on developers to start playing around with their ideas or even build an MVP. They can simply describe it in English as they would to a software house to get what they want. This could well drive down the costs of entrepreneurship as you would no longer need developers to start.
What does that mean to developers, though? Will they become obsolete? Not at all. Instead,they will move higher up the stack. Their primary job is to communicate with the machine to make it do the things that the developer wants. With GPT-3 and similar generative models, that process will happen much faster. New programming languages emerge all the time for a reason: to make programming certain tasks easier and smoother. Generative language models can help build a new generation of programming languages which will power up developers to do incredible things much faster.
All in all, the impact of GPT-3 over the next five years is likely to be increasingly democratized technology. These tools will become cheaper and more accessible to anyone, just like the widespread access to the Internet did 20 years ago.
Regardless of the exact form it takes, with GPT-3, the future of technology definitely looksexciting.
P.S. If you want to test models similar to GPT-3 right now for yourself, visit Contentyze, a content generation platform Im building with my team.
Read the original here:
Why GPT-3 Heralds a Democratic Revolution in Tech - Built In
- Machine Learning Answers: Facebook Stock Is Down 20% In A Month, What Are The Chances It'll Rebound? - Trefis - September 22nd, 2020
- Machine Learning in Education Market Incredible Possibilities, Growth Analysis and Forecast To 2025 - The Daily Chronicle - September 22nd, 2020
- Proximity matters: Using machine learning and geospatial analytics to reduce COVID-19 exposure risk - Healthcare IT News - September 22nd, 2020
- Global Machine Learning Market Tends To Show Steady Growth Post Pandemic With Regional Overview and Top Key Players - Verdant News - September 22nd, 2020
- PREDICTING THE OPTIMUM PATH - Port Strategy - September 22nd, 2020
- AI/ML Remains The Most In-Demand Tech Skill Post COVID - Analytics India Magazine - September 22nd, 2020
- Panalgo Brings the Power of Machine-Learning to the Healthcare Industry Via Its IHD Software - AiThority - September 15th, 2020
- Microchip Partners with Machine-Learning (ML) Software Leaders to Simplify AI-at-the-Edge Design Using its 32-Bit Microcontrollers (MCUs) - EE Journal - September 15th, 2020
- What is 'custom machine learning' and why is it important for programmatic optimisation? - The Drum - September 15th, 2020
- PODCAST: NVIDIA's Director of Data Science Talks Machine Learning for Airlines and Aerospace - Aviation Today - September 15th, 2020
- The Use of Machine Learning to Forecast Progression to Advanced AMD - DocWire News - September 15th, 2020
- How Can Machine Learning Help the Teaching Profession? - FE News - September 15th, 2020
- Global Machine Learning in Automobile Market: Development History, Current Analysis and Estimated Forecast to 2024 - The Market Correspondent - September 15th, 2020
- Using machine learning to organize the chemical diversity - Tech Explorist - September 15th, 2020
- Dashboard AI Announces Its Technology Vision for the Foodservice and Hospitality Industry - PRNewswire - September 15th, 2020
- Alfa Releases Second Paper on AI, Using Machine Learning in the Wild - Monitor Daily - September 10th, 2020
- Combatting COVID-19 misinformation with machine learning (VB Live) - VentureBeat - September 10th, 2020
- This artist used machine learning to create realistic portraits of Roman emperors - The World - September 10th, 2020
- Domino Data Lab Named a Leader in Notebook-Based Predictive Analytics and Machine Learning Evaluation by Global Research Firm - Business Wire - September 10th, 2020
- Demonstration Of What-If Tool For Machine Learning Model Investigation - Analytics India Magazine - September 10th, 2020
- RXA to Participate in 2nd Annual A2.AI Conference focused on Machine Learning & Applied AI - PR Web - September 10th, 2020
- 50 Data Science and Analysts Jobs That Opened Just Last Week - Analytics India Magazine - September 10th, 2020
- FSS Launches Next Gen Recon with Machine Learning and Cloud Support - TechGenyz - September 10th, 2020
- Getting to the heart of machine learning and complex humans - The Irish Times - August 28th, 2020
- Global Machine Learning Courses Market Trends, Key Driven Factors, Segmentation And Forecast To 2020-2026 - The Scarlet - August 28th, 2020
- AI and Machine Learning Network Fetch.ai Partners Open-Source Blockchain Protocol Waves to Conduct R&D on DLT - Crowdfund Insider - August 28th, 2020
- UT Austin Selected as Home of National AI Institute Focused on Machine Learning - UT News | The University of Texas at Austin - August 26th, 2020
- Participation-washing could be the next dangerous fad in machine learning - MIT Technology Review - August 26th, 2020
- The Role of Artificial Intelligence and Machine Learning in the... - Insurance CIO Outlook - August 26th, 2020
- Machine Learning Artificial intelligence Market Size and Growth By Leading Vendors, By Types and Application, By End Users and Forecast to 2020-2027 -... - August 26th, 2020
- Air Force Taps Machine Learning to Speed Up Flight Certifications - Nextgov - August 26th, 2020
- What is AutoML and Why Should Your Business Consider It - BizTech Magazine - August 26th, 2020
- Chatbots Are Machine Learning Their Way To Human Language - Forbes - August 26th, 2020
- Explainable AI: From the peak of inflated expectations to the pitfalls of interpreting machine learning models - ZDNet - August 26th, 2020
- Focusing on ethical AI in business and government - FierceElectronics - August 26th, 2020
- Amazon's Machine Learning University To Make Its Online Courses Available To The Public - Analytics India Magazine - August 14th, 2020
- Watch 3 Videos from Coursera's New "Machine Learning for Everyone" - Machine Learning Times - machine learning & data science news - The... - August 14th, 2020
- PhD Research Fellowship in Machine Learning for Cognitive Power Management job with NORWEGIAN UNIVERSITY OF SCIENCE & TECHNOLOGY - NTNU | 219138 -... - August 14th, 2020
- Machine learning is pivotal to every line of business, every organisation must have an ML strategy - BusinessLine - August 14th, 2020
- CORRECTING and REPLACING Anyscale Hosts Inaugural Ray Summit on Scalable Python and Scalable Machine Learning - Yahoo Finance - August 14th, 2020
- BMW, Red Hat, and Malong Share Insights on AI and Machine Learning During Transform 2020 - ENGINEERING.com - August 14th, 2020
- Algorithm created by deep learning finds potential therapeutic targets throughout the human genome - National Science Foundation - August 14th, 2020
- Ensighten Launches Client-Side Threat Intelligence Initiative and Invests in Machine Learning - WFMZ Allentown - August 6th, 2020
- Hey software developers, youre approaching machine learning the wrong way - The Next Web - August 6th, 2020
- Introducing The AI & Machine Learning Imperative - MIT Sloan - August 6th, 2020
- Who Does the Machine Learning and Data Science Work? - Customer Think - August 6th, 2020
- Artificial Intelligence and Machine Learning Path to Intelligent Automation - Embedded Computing Design - August 6th, 2020
- Blacklight Solutions Unveils Software to Simplify Business Analytics with AI and Machine Learning - PRNewswire - August 6th, 2020
- AI is learning when it should and shouldnt defer to a human - MIT Technology Review - August 6th, 2020
- Moderna Announced Partnership With Amazon Web Services for Their Analytics and Machine Learning Services - Science Times - August 6th, 2020
- Surprisingly Recent Galaxy Discovered Using Machine Learning May Be the Last Generation Galaxy in the Long Cosmic History - SciTechDaily - August 6th, 2020
- STMicroelectronics Releases STM32 Condition-Monitoring Function Pack Leveraging Tools from Cartesiam for Simplified Machine Learning - ELE Times - August 6th, 2020
- Machine Learning Reveals What Makes People Happy In A Relationship - Forbes - August 4th, 2020
- Benefits Of AI And Machine Learning | Expert Panel | Security News - SecurityInformed - August 4th, 2020
- Preparing new machine learning models used to take weeks Activeloop teams up with NVIDIA to reduce that time to hours - MENAFN.COM - August 4th, 2020
- IoT automation trend rides the next wave of machine learning, Big Data - Urgent Communications - August 4th, 2020
- Decoding Practical Problems and Business Implications of Machine Learning - Analytics Insight - August 4th, 2020
- Artificial Intelligence and Machine Learning Industry 2020 Market Manufacturers Analysis, Share, Size, Growth, Trends and Research Report 2026 -... - August 4th, 2020
- Could this software help users trust machine learning decisions? - C4ISRNet - July 27th, 2020
- Top Five Data Privacy Issues that Artificial Intelligence and Machine Learning Startups Need to Know - insideBIGDATA - July 27th, 2020
- COVID-19 Impacts: Machine Learning Market will Accelerate at a CAGR of about 39% through 2020-2024 | The Increasing Adoption of Cloud-based Offerings... - July 27th, 2020
- Deep learning's role in the evolution of machine learning - TechTarget - July 1st, 2020
- 2 books to deepen your command of python machine learning - TechTalks - July 1st, 2020
- What I Learned From Looking at 200 Machine Learning Tools - Machine Learning Times - machine learning & data science news - The Predictive... - July 1st, 2020
- Protecting inventions which use Machine Learning and Artificial Intelligence - Lexology - July 1st, 2020
- Machine learning finds use in creating sharper maps of 'ecosystem' lines in the ocean - Firstpost - July 1st, 2020
- Fake data is great data when it comes to machine learning - Stacey on IoT - July 1st, 2020
- Decisions and NLP Logix Announce Partnership to bring the Power of Machine Learning to Business Process Management - Benzinga - July 1st, 2020
- Machine Learning in Medical Imaging Market Strategies and Insight Driven Transformation 2020-2030 - Cole of Duty - July 1st, 2020
- Impact of COVID-19 Outbreak on Artificial Intelligence and Machine Learning Market to Witness AIBrain, Amazon, Anki, CloudMinds - Cole of Duty - July 1st, 2020
- Machine Learning Market Projected to Register 43.5% CAGR to 2030 Intel, H2Oai - 3rd Watch News - July 1st, 2020
- Learn the business value of AI's various techniques - TechTarget - July 1st, 2020
- Machine Learning As A Service In Manufacturing Market Augmented Expansion to Be Registered by 2018-2023 - 3rd Watch News - July 1st, 2020
- COVID 19 Impact on Machine Learning in Medicine Market Outlook 2020 Industry Size, Top Key Manufacturers, Growth Insights, Demand Analysis and... - July 1st, 2020
- Machine learning algorithm from RaySearch enhances workflow at Swedish radiation therapy clinic - DOTmed HealthCare Business News - July 1st, 2020
- What a machine learning tool that turns Obama white can (and cant) tell us about AI bias - The Verge - June 25th, 2020
- AI and Machine Learning Are Changing Everything. Here's How You Can Get In On The Fun - ExtremeTech - June 25th, 2020
- SLAM + Machine Learning Ushers in the "Age of Perception - Robotics Business Review - June 25th, 2020
- Googles new ML Kit SDK keeps all machine learning on the device - SlashGear - June 25th, 2020
- Machine Learning vs Predictive Analytics: Are they same? - Analytics Insight - June 25th, 2020