Category Archives: Machine Learning
AI In Drug Discovery – Food and Drugs Law – UK – Mondaq
To print this article, all you need is to be registered or login on Mondaq.com.
Developing new or more effective drugs for treating medicalconditions can revolutionise care, and drug discovery is a hugepart of the business of pharmaceutical companies. However, findingwhich drugs are effective for treating which conditions isdifficult. Identifying and screening candidate drugs is typicallyextremely time-consuming, which makes the search for new drugsslow, uncertain, and very expensive.
In modern science, this is not for lack of data. Plenty of dataexists on how small molecules interact with biological systems suchas proteins. However, sorting through all this data to findpromising combinations of molecules and biological pathways totreat particular conditions is very slow. Machine learning offers away to overcome this problem.
We reported recently on Alphafold a machine-learning toolcapable of predicting protein structures with much greaterreliability than previous tools. Other programs already exist thatcan predict the structures of small molecules, which are mucheasier to determine from their chemical composition than thestructures of proteins. Based on the predicted structures ofproteins and small molecules, machine-learning can predict theirinteractions, and work through libraries of molecules to identifycandidate drugs much more quickly than would be possible with humaneffort alone.
This type of processing can identify entirely novel drugs, butmay also be used to identify new applications of existing drugs.Identifying new uses of existing drugs can be particularlyvaluable, since manufacturing capacity and detailed data on sideeffects may already exist that can allow the drug to more rapidlybe repurposed to treat a new condition.
Machine learning can not only identify molecules likely tointeract with a target protein, but may also be able to extrapolateproperties such as toxicity and bio-absorption using data fromother similar molecules. In this way, machine-learning algorithmscould also effectively carry out some of the early stages of drugscreening in silico, thereby reducing the need for expensive andtime-consuming laboratory testing.
Other applications of machine learning in drug discovery includepersonalised medicine. A major problem with some drugs is thevarying response of different individuals to the drug, both interms of efficacy and side-effects. Some patients with chronicconditions such as high blood pressure may spend months or yearscycling through alternative drugs to find one which is effectiveand has acceptable side effects. This can represent an enormouswaste of physician time and create significant inconvenience forthe patient. Using data on the responses of thousands of otherpatients to different drugs, machine learning can be used topredict the efficacy of those drugs for specific individuals basedon genetic profiling or other biological markers.
Identifying candidate drugs as discussed above relies on knowingwhich biological target it is desirable to affect, so thatmolecules can be tested for their interaction with relevantproteins. However, at an even higher level, machine learningtechniques may allow the identification of entirely novelmechanisms for treating medical conditions.
Many studies exist in which participants have their genetic datasequenced, and correlated with data on a wide variety of differentphenotypes. These studies are often used to try to identify geneticfactors that affect an individual's chance of developingdisease. However, machine learning techniques can also identifycorrelations between medical conditions and other measurableparameters, such as expression of certain proteins or levels ofparticular hormones. If plausible biological pathways can bedetermined using these correlations, this could even lead to theidentification of entirely new mechanisms by which certainconditions could be treated.
Examples of AI-based drug discovery already exist in the realworld, with molecules identified using AI methods having enteredclinical trials. Numerous companies are using AI technology toidentify potential new drugs and predict their efficacy forindividual patients. Some estimates suggest that over 2 billion USDin investment funding was raised by companies in this technologyarea in the first half of 2021 alone. As with any technology,patents held by these companies allow them to protect theirintellectual property and provide security for them and theircommercial partners.
Machine learning excels at identifying patterns and correlationsin huge data sets. Exploiting this ability for drug discovery hasthe potential to dramatically improve healthcare outcomes forpatients, and streamline the unwieldy and expensive process ofdeveloping new treatments. We may stand on the threshold of a newera of personalised medicine and rapid drug development.
The content of this article is intended to provide a generalguide to the subject matter. Specialist advice should be soughtabout your specific circumstances.
POPULAR ARTICLES ON: Food, Drugs, Healthcare, Life Sciences from UK
Goodwin Procter LLP
2021 was a banner year for the women's health and wellness industry as global venture capital investment in FemTech companies surpassed $1B for the first time.
Arnold & Porter
In December 2020, we posted about the MHRA's draft guidance on randomised controlled trials generating real-world evidence (RWE) to support regulatory decisions.
Read more from the original source:
AI In Drug Discovery - Food and Drugs Law - UK - Mondaq
MLPDS Machine Learning for Pharmaceutical Discovery and Synthesis …
is a collaboration between the pharmaceutical and biotechnology industries and the departments of Chemical Engineering, Chemistry, and Computer Science at the Massachusetts Institute of Technology. This collaboration will facilitate the design of useful software for the automation of small molecule discovery and synthesis.
The MIT Consortium,Machine Learning for Pharmaceutical Discovery and Synthesis(MLPDS), bringstogether computer scientists, chemical engineers, and chemists from MIT with scientists from membercompaniesto create new data science and artificial intelligence algorithms along with tools to facilitate thediscovery and synthesis of new therapeutics. MLPDS educates scientists and engineers to work effectively atthe data science/chemistry interface and provides opportunities for member companies and MIT tocollectively create, discuss, and evaluate new advances in data science for chemical and pharmaceuticaldiscovery, development, and manufacturing.
Specific research topics within the consortium include synthesis planning; prediction of reaction outcomes,conditions, and impurities; prediction of molecular properties; molecular representation, generation, andoptimization (de novo design); and extraction and organization of chemical information. The algorithms aredeveloped and validated on public data and then transferred to member companies for application toproprietary data. All members share intellectual property and royalty free access to all developments. MITendeavors to make tool development and transfer successful through one-on-one meetings andteleconferences with individual member companies, Microsoft Teams channels, GitLab software repositories,and consortium face-to-face meetings and teleconferences.
More here:
MLPDS Machine Learning for Pharmaceutical Discovery and Synthesis ...
Leveraging machine learning processes to revolutionize the adoption of AI – Express Computer
By Amaresh Tripathy, Global Analytics Business Leader, Genpact
Since the rise of digitalization in the post pandemic world, the role of Artificial Intelligence (AI) and Machine Learning (ML) in driving digital business transformation has greatly increased. Enterprise leaders are accelerating digital initiatives at an unprecedented rate across industries, transforming how people live and work. However, as these programmes take shape, it is observed that only around half of all AI proofs of concept make it to production. For most teams, realizing their AI vision is still a long way off.
The push to move to cloud, as well as the expanding number of machine learning models, that witnessed tremendous growth during the pandemic, is projected to continue in the future. However, while operationalizing Artificial Intelligence, it has been found that merely 27% of the projects piloted by organizations successfully move to production.
What is Machine learning Operations all about?Machine learning operations (MLOps) is all about how to effectively manage data scientists and operations resources to allow for successful development, deployment, and monitoring of models. Simply put, MLOps assist teams in developing, deploying, monitoring, and scaling AI and ML models in a consistent manner, reducing the risks associated with not having a framework for long-term innovation. Consider MLOps to be a success formula.
The challenge The disparity between what AI/ML is used for at present and its potential usage, stems from a number of problems. These are largely related to model building, iteration, deployment, and monitoring. If AI/ML is to alter the global corporate landscape, these concerns must be solved. Organizations that have already begun their path to operationalize AI/ML or are generating Proofs of Concept (PoC) might avoid some of these pitfalls by proactively incorporating best-practices in MLOps to enable smooth model development and addressing scaling issues.
Worse still, organizations spend precious time and resources monitoring and retraining models. Successful machine learning experiments are difficult to duplicate, and data scientists lack access to the technical infrastructure required to develop.
Paving the way to implementationThe development of a Machine Learning model often begins with a business objective, which can be as simple as minimizing fraudulent transactions to less than 0.1 percent or even being able to recognize peoples faces in a photograph on social networking platforms. Additionally, business objectives can also include performance targets, technical infrastructure requirements, and financial constraints; all of which can be represented as key performance indicators, or KPIs, which further allow the business performance of ML models in production to be monitored.
MLOps help ML-based solutions get into production faster through automated model training and retraining processes, as well as continuous integration and continuous delivery strategies for delivering and upgrading Machine Learning pipelines.
MLOps practices and framework allow data engineers to design and build automated data pipelines, data ops platforms, and automated data feedback loops for model improvement, resolving more than 50 issues related to the lack of clean, regulated, governed, and monitored data needed to build production-ready models.
Way forward: The future of MLOpsAccording to our research, many organizations are keen to have centralized ML Operations in the future, as opposed to the current de-centralized approach. The benefit of this type of centralized learning is that the model can generalize based on data from a group of devices and thus work with other compatible devices immediately. Centralized learning also implies that data can explain all the differences between devices and their environments.
MLOps, while still an uncharted territory for many, is quickly becoming a necessity for businesses across industries, with the hope that it makes the business more dependable, scalable and efficient. If the benefits of AI are to be realized, the models that increasingly drive business decisions must follow suit. For years, AI has been optimized through DevOps in the way software is built, run, and maintained, and it is now time to do the same for Machine Learning. It is critical to make AI work at scale with MLOp.
Advertisement
See the rest here:
Leveraging machine learning processes to revolutionize the adoption of AI - Express Computer
Is logistic regression the COBOL of machine learning? – Analytics India Magazine
Joseph Berkson developed logistic regression as a general statistical model in 1944. Today, logistic regression is one of the main pillars of machine learning. From predicting Trauma and Injury Severity Score (TRISS) to sentiment analysis of movie reviews, logistic regression has umpteen applications in ML.
In a recent tweet, Bojan Tunguz, senior software engineer at NVIDIA, compared logistic regression to COBOL, a high-level programming language used for business applications.
It would be great if we could replace all of the logistic regressions with more advanced algos, but realistically we will never completely get rid of them, said Bojan.
COBOL first gained currency in 1970 when it became the de-facto programming language for business applications in mainframe computers around the world.
COBOLs relevance is chalked up to its simplicity, ease of use and portability.
Logistic regression is a simple classification algorithm used to model the probability of a discrete outcome from a given set of input variables. LR is used in supervised learning for binary classification problems.
Pierre Franois Verhulst published the first logistic function in 1838. Logistic regression was used in the biological sciences in the early twentieth century.
Source: dataaspirant.com
After Verhulsts initial discovery of logistic function, the most notable discoveries were the probit model, developed by Chester Ittner Bliss in 1934 and maximum likelihood estimation by Ronald Fisher in 1935.
In 1943, Wilson and Worcester used the logistic model in bioassay which was the first known application of its kind. In 1973 Daniel McFadden connected the multinomial logit to the theory of discrete choice, specifically Luces choice axiom. This gave a theoretical foundation for logistic regression, and earned McFadden a Nobel prize in 2000.
According to the global survey by Micro Focus, COBOL is viewed as strategic by 92 % of respondents.
Key findings of the Micro Focus COBOL Surveys include:
The importance of different AI/ML topics in organisations worldwide (2019). Source: Statista
Bojan Tunguzs tweet garnered both for and against responses.
While many said a simple solution that works should not be messed with, the opposite camp said complex algorithms like XGBoost provide better results.
Andreu Mora, an ML and Data science expert at Adyen payments, said: If a simple algorithm gets you a good performance it might not be a wise move to increase operational work by 500% for a 5% performance uplift.
To this, Bojan replied: Depends on the use case. If a 5% improvement in performance can save you $5B, then you totally should consider it.
Amr Malik, a research fellow at Fast.ai, said: For this scenario to be true, youd need to be supporting a $100 Billion dollar business operation with LR based models. Thatd be a gutsy bet on a really big farm.
We have picked the best responses from the tweet thread:
On LinkedIn, Damien Benveniste, an ML tech lead at Meta AI, said he never uses algorithms like logistic regression, Naive Bayes, SVM, LDA, KNN, Feed Forward Neural Network, etc. and relies only on XGBoost.
Read more here:
Is logistic regression the COBOL of machine learning? - Analytics India Magazine
CFPB Releases a Warning But No Helpful Guidance on Machine Learning Model Adverse Action Notices – Lexology
On May 26, the Consumer Financial Protection Bureau (CFPB or Bureau) announced that federal anti-discrimination law requires companies to explain to applicants the specific reasons for denying an application for credit or taking other adverse actions, even if the creditor is relying on credit models using complex algorithms.
In a corresponding Consumer Financial Protection Circular published the same day, the CFPB started with the question, When creditors make credit decisions do these creditors need to comply with the Equal Credit Opportunity Acts (ECOA) requirement to provide a statement of specific reasons to applicants against whom adverse action is taken?
Yes, the CFPB confirmed. Per the Bureaus analysis, both ECOA and Regulation B require creditors to provide statements of specific reasons to applicants when adverse action is taken. The CFPB is especially concerned with something called black-box models decisions based on outputs from complex algorithms that may make it difficult to accurately identify the specific reasons for denying credit or taking other adverse actions.
This most recent circular asserts that federal consumer financial protection laws and adverse action requirements should be enforced, regardless of the technology used by creditors, and that creditors cannot justify noncompliance with ECOA based on the mere fact that the technology they use to evaluate credit applications is too complicated, too opaque in its decision-making, or too new.
The Bureaus statements are hardly novel. Regulation B requires adverse action notices and does not have an exception for machine learning models, or any other kind of underwriting decision-making for that matter. Its difficult to understand why the Bureau thought it was necessary to restate such a basic principle, but what is even more difficult to understand is why the Bureau has not provided any guidance on the appropriate method for deriving adverse action reasons for machine learning models. The official commentary to Regulation B provides specific adverse action logic applicable to logistic regression models, but the Bureau noted in a July 2020 blog post that there was uncertainty about the most appropriate method to do so with a machine learning model. That same blog post even stated that the Bureau would consider resolving this uncertainty by amending Regulation B or its official commentary. A few months later, the Bureau hosted a Tech Sprint on adverse action notices during which methods for deriving adverse action reasons from machine learning models were specifically presented to the Bureau. Now, a year and half later, the Bureau has still declined to provide any such guidance, and the May 26 announcement simply emphasizes and perpetuates the same uncertainty that the Bureau itself recognized in 2020, without offering any guidance or solution whatsoever. It is disappointing, to say the least.
View original post here:
CFPB Releases a Warning But No Helpful Guidance on Machine Learning Model Adverse Action Notices - Lexology
AI and machine learning are improving weather forecasts, but they won’t replace human experts – The Conversation Indonesia
A century ago, English mathematician Lewis Fry Richardson proposed a startling idea for that time: constructing a systematic process based on math for predicting the weather. In his 1922 book, Weather Prediction By Numerical Process, Richardson tried to write an equation that he could use to solve the dynamics of the atmosphere based on hand calculations.
It didnt work because not enough was known about the science of the atmosphere at that time. Perhaps some day in the dim future it will be possible to advance the computations faster than the weather advances and at a cost less than the saving to mankind due to the information gained. But that is a dream, Richardson concluded.
A century later, modern weather forecasts are based on the kind of complex computations that Richardson imagined and theyve become more accurate than anything he envisioned. Especially in recent decades, steady progress in research, data and computing has enabled a quiet revolution of numerical weather prediction.
For example, a forecast of heavy rainfall two days in advance is now as good as a same-day forecast was in the mid-1990s. Errors in the predicted tracks of hurricanes have been cut in half in the last 30 years.
There still are major challenges. Thunderstorms that produce tornadoes, large hail or heavy rain remain difficult to predict. And then theres chaos, often described as the butterfly effect the fact that small changes in complex processes make weather less predictable. Chaos limits our ability to make precise forecasts beyond about 10 days.
As in many other scientific fields, the proliferation of tools like artificial intelligence and machine learning holds great promise for weather prediction. We have seen some of whats possible in our research on applying machine learning to forecasts of high-impact weather. But we also believe that while these tools open up new possibilities for better forecasts, many parts of the job are handled more skillfully by experienced people.
Today, weather forecasters primary tools are numerical weather prediction models. These models use observations of the current state of the atmosphere from sources such as weather stations, weather balloons and satellites, and solve equations that govern the motion of air.
These models are outstanding at predicting most weather systems, but the smaller a weather event is, the more difficult it is to predict. As an example, think of a thunderstorm that dumps heavy rain on one side of town and nothing on the other side. Furthermore, experienced forecasters are remarkably good at synthesizing the huge amounts of weather information they have to consider each day, but their memories and bandwidth are not infinite.
Artificial intelligence and machine learning can help with some of these challenges. Forecasters are using these tools in several ways now, including making predictions of high-impact weather that the models cant provide.
In a project that started in 2017 and was reported in a 2021 paper, we focused on heavy rainfall. Of course, part of the problem is defining heavy: Two inches of rain in New Orleans may mean something very different than in Phoenix. We accounted for this by using observations of unusually large rain accumulations for each location across the country, along with a history of forecasts from a numerical weather prediction model.
We plugged that information into a machine learning method known as random forests, which uses many decision trees to split a mass of data and predict the likelihood of different outcomes. The result is a tool that forecasts the probability that rains heavy enough to generate flash flooding will occur.
We have since applied similar methods to forecasting of tornadoes, large hail and severe thunderstorm winds. Other research groups are developing similar tools. National Weather Service forecasters are using some of these tools to better assess the likelihood of hazardous weather on a given day.
Researchers also are embedding machine learning within numerical weather prediction models to speed up tasks that can be intensive to compute, such as predicting how water vapor gets converted to rain, snow or hail.
Its possible that machine learning models could eventually replace traditional numerical weather prediction models altogether. Instead of solving a set of complex physical equations as the models do, these systems instead would process thousands of past weather maps to learn how weather systems tend to behave. Then, using current weather data, they would make weather predictions based on what theyve learned from the past.
Some studies have shown that machine learning-based forecast systems can predict general weather patterns as well as numerical weather prediction models while using only a fraction of the computing power the models require. These new tools dont yet forecast the details of local weather that people care about, but with many researchers carefully testing them and inventing new methods, there is promise for the future.
There are also reasons for caution. Unlike numerical weather prediction models, forecast systems that use machine learning are not constrained by the physical laws that govern the atmosphere. So its possible that they could produce unrealistic results for example, forecasting temperature extremes beyond the bounds of nature. And it is unclear how they will perform during highly unusual or unprecedented weather phenomena.
And relying on AI tools can raise ethical concerns. For instance, locations with relatively few weather observations with which to train a machine learning system may not benefit from forecast improvements that are seen in other areas.
Another central question is how best to incorporate these new advances into forecasting. Finding the right balance between automated tools and the knowledge of expert human forecasters has long been a challenge in meteorology. Rapid technological advances will only make it more complicated.
Ideally, AI and machine learning will allow human forecasters to do their jobs more efficiently, spending less time on generating routine forecasts and more on communicating forecasts implications and impacts to the public or, for private forecasters, to their clients. We believe that careful collaboration between scientists, forecasters and forecast users is the best way to achieve these goals and build trust in machine-generated weather forecasts.
See the original post here:
AI and machine learning are improving weather forecasts, but they won't replace human experts - The Conversation Indonesia
Incorporation of machine learning and deep neural network approaches into a remote sensing-integrated crop model for the simulation of rice growth |…
Study locations and rice data
The ML and DNN models were developed for the rice growing areas in the entire geographic regions of Cheorwon and Paju in South Korea (Fig.4). Then, the parameterised ML and DNN models were evaluated for the representative rice growing areas of Gimje, South Korea and Pyeongyang, North Korea. Cheorwon and Paju were selected as these areas are typical rice cultivation regions in the central portion of the Korean peninsula. The paddy rice cultivation regions in Cheorwon and Paju have areas of 10,169 and 6,625ha, respectively, representing 80.4% and 62.6% of the total staple croplands for each region, according to the Korean Statistical Information Service, KOSIS (https://kosis.kr/).
Study location boundary maps of (a) Cheorwon, (b) Paju, (c) Gimje in South Korea and (d) Pyeongyang in North Korea.
The leading rice cultivar in Cheorwon and Paju was Odae (bred by NICS in 1983), cultivated in more than 80% of the paddy fields during the study period, according to KOSIS. Rice seedlings were transplanted in these areas between May 15 and 20, deemed as the ideal transplanting period.
We used the temporal profiles of NDVI from the Terra MODIS MOD09A1 surface reflectance 8-day product with a spatial resolution of 500m, which were employed for the ML and DNN model input variable. This product is the composited imagery by selecting the best pixels considering the cloud and solar zenith during eight days33. It is essential to secure reliable and continuous phenological NDVI data for determining crop yield in monsoon regions like the current study area concerning input variables for the process-based crop model. Therefore, the cloud-contaminated pixels were interpolated with other poor quality pixels caused by aerosol quantity or cloud shadow using the spline interpolation algorithm during the rice-growing season to improve data quality during the monsoon season. This approach has been widely used in time series satellite imagery for interpolation34,35,36. The criteria for poor quality pixels for interpolation were determined from the 16-bit quality assurance (QA) flags from the MOD09A1 product33.
Furthermore, we estimated the incoming solar radiation on the surface (insolation) obtained from the COMS Meteorological Imager (MI). Insolation reflects the energy source of photosynthesis for the crop canopies. We adopted a physical model to estimate solar radiation by considering atmospheric effects such as aerosol, water vapour, ozone, and Rayleigh scattering37,38,39,40,41. Before estimating the solar radiation from the physical model, we classified clear and cloudy sky conditions because cloud effects should be considered for their high attenuation influences. If the pixel image was assigned as a clear sky condition, atmospheric parameterisations were performed for direct and diffuse irradiance owing to the effects of atmospheric constituents and solar-target-satellite sensor geometry40,42,43,44. If the pixel images were considered as under cloudy conditions, the cloud attenuation was calculated using a cloud factor for visible reflectance and the solar zenith angle42. Finally, the estimated solar radiation from COMS MI was used as one of the main input parameters of the RSCM system. Comprehensive descriptions of those parameters used for the physical model can be referenced from earlier studies41,43.
The maximum and minimum air temperature data were obtained from the Regional Data Assimilation and Prediction System (RDAPS) provided by the Korea Meteorological Administration (KMA, https://www.kma.go.kr). The spatial resolution of the RDAPS is 12km, and it is composed of 70 vertical levels up to about 80km. The global data assimilation and prediction system is provided at 3-h intervals for the Asian regions, and forecasts are performed four times a day (00, 06, 12, and 18 UTC) for 87h. In addition, the system is operated in a 6-h interval analysis-prediction-circulation system using the four-dimensional variational data assimilation45. The weather datasets were resampled to a spatial resolution of 500m using the nearest neighbour method that does not change the existing values to match the MODIS imagery.
The current study employed the RSCM to incorporate an ML and DNN procedure and then simulate rice growths and yields (Supplementary Fig. S1). We integrated an ML and DNN regressor into the RSCM-rice system based on the investigation of the ML or DNN regressors described in the following subsection. The ML or DNN scheme was implemented to improve the mathematical regression approach for the RS-based VIs and LAI relationships, as described below.
RSCM is a process-based crop model designed to integrate remotely sensed data, allowing crop modellers to simulate and monitor potential crop growth6. This model can accept RS data as input to perform its within-season calibration procedure5, wherein the simulated LAI values are compared to the corresponding observed values. Four different parameters (that is, L0, a, b, and c) are utilised in the within-season procedure to define the crop-growth processes based on the optimisation of LAI using the POWELL procedure46. In addition, these parameters can be calibrated using the Bayesian method to obtain acceptable values with a prior distribution that was selected based on the estimates from earlier studies6,47. The current research project applied consistent initial conditions and parameters to calibrate the RSCM-rice system.
The ML models investigated in this study were Polynomial regression, Ridge, Least Absolute Shrinkage and Selection Operator (LASSO), Support Vector Regression (SVR), RF, Extra Trees (ET), Gradient Boosting (GB), Histogram-based Gradient Boosting (HGB), Extreme Gradient Boosting (XGB), and Light Gradient Boosting machine regression (LightGB) regressors. These models are implemented in scikit-learn (https://scikit-learn.org/), while the DNN model (Supplementary Fig. S4) is implemented in Keras (https://keras.io/), which are achievable on Python (https://www.python.org/).
The Polynomial regression model is a particular regression model to overcome the limitations of simple linear regression by estimating the relationship with the Nth degree polynomial. The Ridge and Lasso additionally use l2-norm and l1-norm as constraints in the existing model. These characteristics of the models show better performance than the conventional linear regression, which uses the least-squares method to find appropriate weights and biases to reduce overfitting48,49.
The SVR allows the definition of the amount of allowable error and finds a hyperplane of higher dimensions to fit the data. The SVR is widely used for classification and numerical prediction and is less overfitting and easier to use than neural networks. However, it takes a long time to build an optimisation model, and it is difficult to interpret the results50.
The RF is an ensemble model that trains multiple decision tree models and aggregates its results. It has good generalisation and performance, is easy to tune parameters, and is less prone to overfitting. On the other hand, memory consumption is higher than in other ML models. Also, it is not easy to expect higher performance improvement even when the amount of training dataset increases. Extra trees increase randomness by randomly splitting each candidate feature in the tree, which can reduce bias and variance51. The difference from the RF is that ET does not use bootstrap sampling but uses the whole origin data when making decision trees. The GB belongs to the boosting series among the RF ensemble models, which combines weak learners to create strong learners with increased performance. Meanwhile, the GB training process is slow and not efficient in overfitting. There are HGB, XGB, and LightGB in the form of the GB that improve performance by increasing the training speed and reducing overfitting. The HGB speeds up the algorithm by grouping each decision tree with a histogram and reducing the number of features. The XGB improves learning speed through parallel processing and is equipped with functions necessary to improve performance compared to the GB, such as regularisation, tree pruning, and early stopping. The LightGBM significantly shortens the training time and decreases memory use by using a histogram-based algorithm without showing a significant difference in predictive performance compared to the XGBoost52.
The DNN increases the predictive power by increasing the hidden layer between the input and the output layers. Non-linear combinations between input variables are possible, feature weighting is performed automatically, and performance tends to increase as the amount of data increases. However, since it is difficult to interpret the meaning of the weights, there is a disadvantage in that the results are also difficult to interpret. In addition, when fewer training datasets are collected, the performance of the ML models mentioned above can be better53.
This study used satellite-based solar radiation and model-based maximum and minimum temperatures to estimate LAI values during the rice-growing seasons on the study sites (Cheorwon, Paju, Gimje, and Pyeongyang) for seven years (20112017), employing the ML and DNN regressors. We reproduced rice LAI values from the MODIS-based NDVI values using the empirical relationship between LAI and NDVI (Supplementary Fig. S2). Cheorwon and Paju datasets were used for the ML and DNN model development, while Gimje and Pyeongyang datasets were employed for the model evaluation. The target LAI variable data used for the model development showed characteristic seasonal and geographical variations (Supplementary Figs. S3 and S4). The model development datasets were divided into train and test sets with a 0.8 and 0.2 ratio using the scikit-learn procedure. All the ML and DNN regressors were trained and tested, obtaining appropriate hyperparameters. Alpha values for the Ridge and Lasso were determined as 0.1 and 0.01 based on a grid search approach with a possible range of values (Supplementary Fig. S5). The activation function employed for the DNN model was the rectified linear unit (ReLU), implementing six fully connected layers with a design of gradual increasing and decreasing units from 100 to 1,000 (Supplementary Fig. S6). The model was performed with a dropout rate of 0.17, the adam optimizer at a learning rate of 0.001, 1,000 epochs, and a batch size of 100. The DNN hyperparameters were determined based on a grid search approach and a trial and error approach, seeking minimum and steady losses for each study region (Supplementary Fig. S7).
We analysed the performance of the ML (that is, RF) and DNN regimes using two statistical indices in Python (https://www.python.org), namely the RMSE and the ME54. This index denotes the comparative scale of the residual variance of simulated data compared to the observed data variance. Furthermore, ME can assess the agreement between the experimental and simulated data, showing how well these data fit through the 1:1 line in a scatter plot. The index value can vary from to 1. We employed normalized ME for advanced interpretation, allowing for the ME measure in simulation estimation approaches used in model evaluation. Thus, ME=1, 0, and correspond to ME=1, 0.5, and 0, respectively. Therefore, the model is considered reliable if the ME value is nearer to 1, whereas the simulated data are considered less dependable if the ME value is close to 0.
Read more from the original source:
Incorporation of machine learning and deep neural network approaches into a remote sensing-integrated crop model for the simulation of rice growth |...
What Hugging Face and Microsofts collaboration means for applied AI – TechTalks
This article is part of our series that explores thebusiness of artificial intelligence
Last week, Hugging Face announced a new product in collaboration with Microsoft called Hugging Face Endpoints on Azure, which allows users to set up and run thousands of machine learning models on Microsofts cloud platform.
Having started as a chatbot application, Hugging Face made its fame as a hub for transformer models, a type of deep learning architecture that has been behind many recent advances in artificial intelligence, including large language models like OpenAI GPT-3 and DeepMinds protein-folding model AlphaFold.
Large tech companies like Google, Facebook, and Microsoft have been using transformer models for several years. But the past couple of years has seen a growing interest in transformers among smaller companies, including many that dont have in-house machine learning talent.
This is a great opportunity for companies like Hugging Face, whose vision is to become the GitHub for machine learning. The company recently secured $100 million in Series C at a $2 billion valuation. The company wants to provide a broad range of machine learning services, including off-the-shelf transformer models.
However, creating a business around transformers presents challenges that favor large tech companies and put companies like Hugging Face at a disadvantage. Hugging Faces collaboration with Microsoft can be the beginning of a market consolidation and a possible acquisition in the future.
Transformer models can do many tasks, including text classification, summarization, and generation; question answering; translation; writing software source code; and speech to text conversion. More recently, transformers have also moved into other areas, such as drug research and computer vision.
One of the main advantages of transformer models is their capability to scale. Recent years have shown that the performance of transformers grows as they are made bigger and trained on larger datasets. However, training and running large transformers is very difficult and costly. A recent paper by Facebook shows some of the behind-the-scenes challenges of training very large language models. While not all transformers are as large as OpenAIs GPT-3 and Facebooks OPT-175B, they are nonetheless tricky to get right.
Hugging Face provides a large repertoire of pre-trained ML models to ease the burden of deploying transformers. Developers can directly load transformers from the Hugging Face library and run them on their own servers.
Pre-trained models are great for experimentation and fine-tuning transformers for downstream applications. However, when it comes to applying the ML models to real products, developers must take many other parameters into consideration, including the costs of integration, infrastructure, scaling, and retraining. If not configured right, transformers can be expensive to run, which can have a significant impact on the products business model.
Therefore, while transformers are very useful, many organizations that stand to benefit from them dont have the talent and resources to train or run them in a cost-efficient manner.
An alternative to running your own transformer is to use ML models hosted on cloud servers. In recent years, several companies launched services that made it possible to use machine learning models through API calls without the need to know how to train, configure, and deploy ML models.
Two years ago, Hugging Face launched its own ML service, called Inference API, which provides access to thousands of pre-trained models (mostly transformers) as opposed to the limited options of other services. Customers can rent Inference API based on shared resources or have Hugging Face set up and maintain the infrastructure for them. Hosted models make ML accessible to a wide range of organizations, just as cloud hosting services brought blogs and websites to organizations that couldnt set up their own web servers.
So, why did Hugging Face turn to Microsoft? Turning hosted ML into a profitable business is very complicated (see, for example, OpenAIs GPT-3 API). Companies like Google, Facebook, and Microsoft have invested billions of dollars into creating specialized processors and servers that reduce the costs of running transformers and other machine learning models.
Hugging Face Endpoints takes advantage of Azures main features, including its flexible scaling options, global availability, and security standards. The interface is easy to use and only takes a few clicks to set up a model for consumption and configure it to scale at different request volumes. Microsoft has already created a massive infrastructure to run transformers, which will probably reduce the costs of delivering Hugging Faces ML models. (Currently in beta, Hugging Face Endpoints is free, and users only pay for Azure infrastructure costs. The company plans a usage-based pricing model when the product becomes available to the public.)
More importantly, Microsoft has access to a large share of the market that Hugging Face is targeting.
According to the Hugging Face blog, As 95% of Fortune 500 companies trust Azure with their business, it made perfect sense for Hugging Face and Microsoft to tackle this problem together.
Many companies find it frustrating to sign up and pay for various cloud services. Integrating Hugging Faces hosted ML product with Microsoft Azure ML reduces the barriers to delivering its products value and expands the companys market reach.
Hugging Face Endpoints can be the beginning of many more product integrations in the future, as Microsofts suite of tools (Outlook, Word, Excel, Teams, etc.) have billions of users and provide plenty of use cases for transformer models. Company execs have already hinted at plans to expand their partnership with Microsoft.
This is the start of the Hugging Face and Azure collaboration we are announcing today as we work together to bring our solutions, our machine learning platform, and our models accessible and make it easy to work with on Azure. Hugging Face Endpoints on Azure is our first solution available on the Azure Marketplace, but we are working hard to bring more Hugging Face solutions to Azure, Jeff Boudier, product director at Hugging Face, told TechCrunch. We have recognized [the] roadblocks for deploying machine learning solutions into production [emphasis mine] and started to collaborate with Microsoft to solve the growing interest in a simple off-the-shelf solution.
This can be extremely advantageous to Hugging Face, which must find a business model that justifies its $2-billion valuation.
But Hugging Faces collaboration with Microsoft wont be without tradeoffs.
Earlier this month, in an interview with Forbes, Clment Delangue, Co-Founder and CEO at Hugging Face, said that he has turned down multiple meaningful acquisition offers and wont sell his business, like GitHub did to Microsoft.
However, the direction his company is now taking will make its business model increasingly dependent on Azure (again, OpenAI provides a good example of where things are headed) and possibly reduce the market for its independent Inference API product.
Without Microsofts market reach, Hugging Faces product(s) will have greater adoption barriers, lower value proposition, and higher costs (the roadblocks mentioned above). And Microsoft can always launch a rival product that will be better, faster, and cheaper.
If a Microsoft acquisition proposal comes down the line, Hugging Face will have to make a tough choice. This is also a reminder of where the market for large language models and applied machine learning is headed.
In comments that were published on the Hugging Face blog, Delangue said, The mission of Hugging Face is to democratize good machine learning. Were striving to help every developer and organization build high-quality, ML-powered applications that have a positive impact on society and businesses.
Indeed, products like Hugging Face Endpoints will democratize machine learning for developers.
But transformers and large language models are also inherently undemocratic and will give too much power to a few companies that have the resources to build and run them. While more people will be able to build products on top of transformers powered by Azure, Microsoft will continue to secure and expand its market share in what seems to be the future of applied machine learning. Companies like Hugging Face will have to suffer the consequences.
Go here to see the original:
What Hugging Face and Microsofts collaboration means for applied AI - TechTalks
Machine Learning Shows That More Reptile Species May Be at Risk of Extinction Than Previously Thought – SciTechDaily
Potamites montanicola, classified as Critically Endangered by automated the assessment method and as Data Deficient by the IUCN Red List of Threatened Species. Credit: Germn Chvez, Wikimedia Commons (CC-BY 3.0)
Machine learning tool estimates extinction risk for species previously unprioritized for conservation.
Species at risk of extinction are identified in the iconic Red List of Threatened Species, published by the International Union for Conservation of Nature (IUCN). A new study presents a novel machine learning tool for assessing extinction risk and then uses this tool to show that reptile species which are unlisted due to lack of assessment or data are more likely to be threatened than assessed species. The study, by Gabriel Henrique de Oliveira Caetano at Ben-Gurion University of the Negev, Israel, and colleagues, was published on May 26th in the journal PLOS Biology.
The IUCNs Red List of Threatened Species is the most comprehensive assessment of the extinction risk of species and informs conservation policy and practices around the world. However, the process for categorizing species is time-consuming, laborious, and subject to bias, depending heavily on manual curation by human experts. Therefore, many animal species have not been evaluated, or lack sufficient data, creating gaps in protective measures.
To assess 4,369 reptile species that were previously unable to be prioritized for conservation and develop accurate methods for assessing the extinction risk of obscure species, these scientists created a machine learning computer model. The model assigned IUCN extinction risk categories to the 40% of the worlds reptiles that lacked published assessments or are classified as DD (Data Deficient) at the time of the study. The researchers validated the models accuracy, comparing it to the Red List risk categorizations.
The authors found that the number of threatened species is much higher than reflected in the IUCN Red List and that both unassessed (Not Evaluated or NE) and Data Deficient reptiles were more likely to be threatened than assessed species. Future studies are needed to better understand the specific factors underlying extinction risk in threatened reptile taxa, to obtain better data on obscure reptile taxa, and to create conservation plans that include newly identified, threatened species.
According to the authors, Altogether, our models predict that the state of reptile conservation is far worse than currently estimated, and that immediate action is necessary to avoid the disappearance of reptile biodiversity. Regions and taxa we identified as likely to be more threatened should be given increased attention in new assessments and conservation planning. Lastly, the method we present here can be easily implemented to help bridge the assessment gap on other less known taxa.
Coauthor Shai Meiri adds, Importantly, the additional reptile species identified as threatened by our models are not distributed randomly across the globe or the reptilian evolutionary tree. Our added information highlights that there are more reptile species in peril especially in Australia, Madagascar, and the Amazon basin all of which have a high diversity of reptiles and should be targeted for extra conservation efforts. Moreover, species-rich groups, such as geckos and elapids (cobras, mambas, coral snakes, and others), are probably more threatened than the Global Reptile Assessment currently highlights, these groups should also be the focus of more conservation attention
Coauthor Uri Roll adds, Our work could be very important in helping the global efforts to prioritize the conservation of species at risk for example using the IUCN red-list mechanism. Our world is facing a biodiversity crisis, and severe man-made changes to ecosystems and species, yet funds allocated for conservation are very limited. Consequently, it is key that we use these limited funds where they could provide the most benefits. Advanced tools- such as those we have employed here, together with accumulating data, could greatly cut the time and cost needed to assess extinction risk, and thus pave the way for more informed conservation decision making.
Reference: Automated assessment reveals that the extinction risk of reptiles is widely underestimated across space and phylogeny by Gabriel Henrique de Oliveira Caetano, David G. Chapple, Richard Grenyer, Tal Raz, Jonathan Rosenblatt, Reid Tingley, Monika Bhm, Shai Meiri and Uri Roll. 26 May 2022, PLOS Biology.DOI: 10.1371/journal.pbio.3001544
Here is the original post:
Machine Learning Shows That More Reptile Species May Be at Risk of Extinction Than Previously Thought - SciTechDaily
Scientists use AI to update data vegetation maps for improved wildfire forecasts | NCAR & UCAR News – University Corporation for Atmospheric…
May 31, 2022 - by Laura Snider
A cabin on the shore of Grand Lake in Colorado, near the area where the East Troublesome Fire burned in 2020. Many of the lodgepole pines in the region were killed by pine beetles. (Image: Don Graham/Flickr)
A new technique developed by the National Center for Atmospheric Research (NCAR) uses artificial intelligence to efficiently update the vegetation maps that are relied on by wildfire computer models to accurately predict fire behavior and spread.
In a recent study, scientists demonstrated the method using the 2020 East Troublesome Fire in Colorado, which burned through land that was mischaracterized in fuel inventories as being healthy forest. In fact the fire, which grew explosively, scorched a landscape that had recently been ravaged by pine beetles and windstorms, leaving significant swaths of dead and downed timber.
The research team compared simulations of the fire generated by a state-of-the-art wildfire behavior model developed at NCAR using both the standard fuel inventory for the area and one that was updated with artificial intelligence (AI). The simulations that used the AI-updated fuels did a significantly better job of predicting the area burned by the fire, which ultimately grew to more than 190,000 acres of land on both sides of the continental divide.
One of our main challenges in wildfire modeling has been to get accurate input, including fuel data, said NCAR scientist and lead author Amy DeCastro. In this study, we show that the combined use of machine learning and satellite imagery provides a viable solution.
The research was funded by the U.S. National Science Foundation, which is NCARs sponsor. The modeling simulations were run at the NCAR-Wyoming Supercomputing Center on the Cheyenne system.
For a model to accurately simulate a wildfire, it requires detailed information about the current conditions. This includes the local weather and terrain as well as the characteristics of the plant matter that provides fuel for the flames whats actually available to burn and what condition its in. Is it dead or alive? Is it moist or dry? What type of vegetation is it? How much is there? How deep is the fuel layered on the ground?
The gold standard of fuel datasets is produced by LANDFIRE, a federal program that produces a number of geospatial datasets including information on wildfire fuels. The process of creating these wildfire fuel datasets is extensive and incorporates satellite imagery, landscape simulation, and information collected in person during surveys. However, the amount of resources necessary to produce them means that, practically speaking, they cannot be updated frequently, and disturbance events in the forest including wildfires, insect infestations, and development can radically alter the available fuels in the meantime.
In the case of the East Troublesome Fire, which began in Grand County, Colorado, and burned east into Rocky Mountain National Park, the most recent LANDFIRE fuel dataset was released in 2016. In the intervening four years, the pine beetles had caused widespread tree mortality in the area.
To update the fuel dataset, the researchers turned to the Sentinel satellites, which are part of the European Space Agencys Copernicus program. Sentinel-1 provides information about surface texture, which can be used to identify vegetation type. (Grass has a very different texture than trees, for example.) And Sentinel-2 provides information that can be used to infer the plants health from its greenness. The scientists fed the satellite data into a machine learning model known as a random forest that they had trained on the U.S. Forest Services Insect and Disease Detection Survey. The survey is conducted annually by trained staff who estimate tree mortality from the air.
The result was that the machine learning model was able to accurately update the LANDFIRE fuel data, turning the majority of the fuels categorized as timber litter or timber understory to slash blowdown, the designation used for forested regions with heavy tree mortality.
The LANDFIRE data is super valuable and provides a reliable platform to build on, DeCastro said. Artificial intelligence proved to be an effective tool for updating the data in a less resource-intensive manner.
To test the effect the updated fuel inventory would have on wildfire simulation, the scientists used a version of NCARs Weather Research and Forecasting model, known as WRF-Fire, which was specifically developed to simulate wildfire behavior.
When WRF-Fire was used to simulate the East Troublesome Fire using the unadjusted LANDFIRE fuel dataset it substantially underpredicted the amount of area the fire would burn. When the model was run again with the adjusted dataset, it was able to predict the area burned with a much greater degree of accuracy, indicating that the dead and downed timber helped fuel the fires spread much more so than if the trees had still been alive.
For now, the machine learning model is designed to update an existing fuel map, and it can do the job quickly (in a matter of minutes). But the success of the project also shows the promise of using a machine learning system to begin regularly producing and updating fuel maps from scratch over large regions at risk from wildfires.
The new research at NCAR is part of a larger trend of investigating possible AI applications for wildfire, including efforts to use AI to more quickly estimate fire perimeters. NCAR researchers are also hopeful that machine learning may be able to help solve other persistent challenges for wildfire behavior modeling. For example, machine learning may be able to improve our ability to predict the properties of the embers generated by a fire (how big, how hot, and how dense) as well as the likelihood that those embers could cause spot fires.
We have so much technology and so much computing power and so many resources at our fingertips to solve these issues and keep people safe, said NCAR scientist Timothy Juliano, a study co-author. Were well positioned to make a positive impact; we just need to keep working on it.
See all News
See the original post here:
Scientists use AI to update data vegetation maps for improved wildfire forecasts | NCAR & UCAR News - University Corporation for Atmospheric...