As a decorated college runner at Notre Dame and then at the University of Tennessee, Dylan Jacobs dabbled with a device that many of his teammates considered indispensable.

But on those rare occasions when Jacobs succumbed to peer pressure and slapped a GPS watch around his wrist, he almost immediately remembered why he had resisted the temptation in the first place.

The runs just felt so much longer, said Jacobs, 23, a three-time N.C.A.A. champion who recently turned pro. That was one of my main problems with it. I wasnt enjoying myself or looking around. Instead, I was kind of looking at the watch every quarter-mile to see how much longer I had left.

GPS watches popular brands are Garmin, Suunto and Coros come equipped with satellite technology and heart rate monitors to produce a buffet of functions. Want to know how far and how fast youve run, or how many milliliters of sweat you dumped in Central Park last weekend? How about your average stride length? Your cadence? The list goes on.

For many, GPS watches are a remarkably useful training tool. But there are other runners, including world-class runners like Jacobs, who have a hard time understanding the fuss. To them, a smorgasbord of data is more hindrance than help. And get this: Some runners dont wear watches at all.

I like to focus more on the feel of everything and not worry too much about the time, Jacobs said.

Heather MacLean, an Olympic 1,500-meter runner, recalled a period of her life when she enjoyed the utility of a GPS watch. As a student at the University of Massachusetts, she grew to understand the value of sleep and more important, that she was not getting enough of it while working in a neuroscience laboratory. So she began using a Garmin Forerunner to monitor her rest and adjust her schedule.

Later, as a first-year pro with Team New Balance Boston, MacLean tried to be consistent about wearing a GPS watch but was hampered by a couple of issues. First, she was always forgetting to charge it.

I would just let it die all the time, and Im super lazy with that kind of stuff, she said.

Second, MacLean realized her watch was draining the fun from her runs. It was especially apparent to her during a low-key stretch when she was simply trying to build fitness.

I hated that every run I went on, I felt like I had to check my pace and my distance and whatever else, she said. So I just decided that I was going to lay off it for a while and switch to a regular watch.

She never went back. MacLean, 28, who now wears an Armitron Dragonfly that she said she picked up for $10 at Walmart, acknowledged that there were certain workouts when a GPS watch would come in handy, like when she did a tempo run by herself. (Tempo runs are faster than easy jogs, and frequently run at a prescribed pace.) But Mark Coogan, her coach, has long prioritized effort over pace, and MacLean logs her training in minutes rather than in miles.

I know Im at the elite level now, so not everything is going to be joyful, MacLean said. But when there are things that bring me a lot of joy, Im going to invest in them. And one of those things is the ability to avoid focusing on my pace during my runs.

Without the pressure of feeling as if she needs to account for every mile or, perish the thought, post her workouts for public inspection on Strava, the exercise-tracking platform MacLean has also gotten better about listening to her body. She has no qualms about bailing on an extra workout if she is feeling beat.

And Ill tell Mark that Im going for a walk instead, MacLean said. And hes like, OK!

Sam Prakel was a high school standout in Versailles, Ohio, when the assistant coach of his cross-country team introduced him to the magic of GPS watches. Prakel invested in one. It was a mistake from the start.

I just started running too fast on all my runs, Prakel said, and it became harder to recover from them because I was so focused on my pace. I learned pretty quickly that it wasnt good for me.

Prakel opted instead for a Timex Ironman, which he wore through his freshman year at the University of Oregon. When the band snapped in his sophomore year, he ordered another. Prakel, 28, has worn the same no-frills watch ever since through his time at Oregon, where he was a five-time all-American, and in more recent years as a pro miler for Adidas. He has never needed to change its battery.

The reigning U.S. indoor champion in the mens 1,500 and 3,000 meters, Prakel has a system that works for him, which is a throwback in a sense. What did any runners do before the advent of GPS watches? They estimated. In Prakels case, a 65-minute run is roughly equivalent to 10 miles and a half-hour jog is good for four miles. He does not need to be precise.

As long as I do the same things every week and keep it consistent, thats all that matters, he said, adding: I feel like Im in a better place when I dont have all that data to worry about.

For some runners, aesthetics also matter. Luke Houser, a junior at the University of Washington who won an N.C.A.A. championship in the mens indoor mile last winter, wears a vintage-inspired Casio with a digital display and a gold metal band. His teammates simply refer to it as the gold Casio.

I just think it looks cool, he said. Ive never been interested in cadence or heart rate, which I dont think is ever that accurate anyway. All you need to know is how you feel and the time. That does the job.

Kieran Lumb, who recently broke his own Canadian record in the mens 3,000 meters, is well aware that he is the type of person who is susceptible to the sweet lure of data.

At the University of British Columbia, Lumb majored in electrical engineering. Later, while running at Washington, he earned a masters degree in information systems. And for the longest time, no one who knew him was surprised that he maintained an Excel spreadsheet to catalog his sleep, workouts and something he called rated perceived fatigue.

Just trying to do a little bit of data science on myself, he said.

The twist is that Lumb, 25, who now runs professionally for the athletics apparel brand On, has not worn a GPS watch since he was a competitive cross-country skier growing up in Canada. He made the switch as a college freshman to a Casio calculator watch that didnt even have a proper lap function for track workouts.

So Id just have to remember all my splits, he said, and it was awesome.

Lumb noted that because many runners are naturally competitive, they can become obsessed with numbers. And the business of making it to the top of the heap as an elite runner can be especially taxing.

As a result, Lumbs coach, Andy Powell, tries to keep things as simple as possible. For Lumb, that has meant ditching his Excel folder in favor of Powells old-school approach: weekly workout sheets that his runners fill out and file in three-ring binders.

Theres something nice about slowing down and writing it by hand that I find almost endearing, Lumb said. Its taken a while for me to be less neurotic, but its liberating.

View post:

Why Some Top Runners Prefer to Train Without a GPS Watch - The New York Times

Data science is a practice that requires technical expertise in machine learning and code development. However, it also demands creativity (for instance, connecting dense numbers and data to real user needs) and lean thinking (like prioritizing the experiments and questions to explore next). In light of these needs, and to continuously innovate and create meaningful outcomes, its essential to adopt processes and techniques that facilitate high levels of energy, drive and communication in data science development.

Pair programming can increase communication, creativity and productivity in data science teams. Pair programming is a collaborative way of working in which two people take turns coding and navigating on the same problem, at the same time, on the same computer connected with two mirrored screens, two mice and two keyboards.

At VMware Tanzu Labs, our data scientists practice pair programming with each other and with our client-side counterparts. Pair programming is more widespread in software engineering than in data science. We see this as a missed opportunity. Lets explore the nuanced benefits of pair programming in the context of data science, delving into three aspects of the data science life cycle and how pair programming can help with each one.

When data scientists pick up a story for development, exploratory data analysis (EDA) is often the first step in which we start writing code. Arguably, among all components of the development cycle that require coding, EDA demands the most creativity from data scientists: The aim is to discover patterns in the data and build hypotheses around how we might be able to use this information to deliver value for the story at hand.

If new data sources need to be explored to deliver the story, we get familiar with them by asking questions about the data and validating what information they are able to provide to us. As part of this process, we scan sample records and iteratively design summary statistics and visualizations for reexamination.

Pairing in this context enables us to immediately discuss and spark a continuous stream of second opinions and tweaks on the statistics and visualizations displayed on the screen; we each build on the energy of our partner. Practicing this level of energetic collaboration in data science goes a long way toward building the creative confidence needed to generate a wider range of hypotheses, and it adds more scrutiny to synthesis when distinguishing between coincidence and correlation.

Based on what we learn about the data from EDA, we next try to summarize a pattern weve observed, which is useful in delivering value for the story at hand. In other words, we build or train a model that concisely and sufficiently represents a useful and valuable pattern observed in the data.

Arguably, this part of the development cycle demands the most science from data scientists as we continuously design, analyze and redesign a series of scientific experiments. We iterate on a cycle of training and validating model prototypes and make a selection as to which one to publish or deploy for consumption.

Pairing is essential to facilitating lean and productive experimentation in model training and validation. With so many options of model forms and algorithms available, balancing simplicity and sufficiency is necessary to shorten development cycles, increase feedback loops and mitigate overall risk in the product team.

As a data scientist, I sometimes need to resist the urge to use a sophisticated, stuffy algorithm when a simpler model fits the bill. I have biases based on prior experience that influence the algorithms explored in model training.

Having my paired data scientist as my data conscience in model training helps me put on the brakes when Im running a superfluous number of experiments, constructively challenges the choices made in algorithm selection and course-corrects me when I lose focus from training prototypes strictly in support of the current story.

In addition to aspects of pair programming that influence productivity in specific components of the development cycle such as EDA and model training/validation, there are also perhaps more mundane benefits of pairing for data science that affect productivity and reproducibility more generally.

Take the example of pipelining. Much of the code written for data science is sequential by nature. The metrics we discover and design in EDA are derived from raw data that requires sequential coding to clean and process. These same metrics are then used as key pieces of information (a.k.a. features) when we build experiments for model training. In other words, the code written to design these metrics is a dependency for the code written for model training. Within model training itself, we often try different versions of a previously trained model (which we have previously written code to build) by exploring different variations of input parameter values to improve accuracy. The components and dependencies described above can be represented as steps and segments in a logical, sequential pipeline of code.

Pairing in the context of pipelining brings benefits in shared accountability driven by a sense of shared ownership of the codebase. While all data scientists know and understand the benefits of segmenting and modularizing code, when coding without a pair, it is easy to slip into a habit of creating overly lengthy code blocks, losing count on similar code being copied-pasted-modified and discounting groups of code dependencies that are only obvious to the person coding. These habits create cobwebs in the codebase and increase risks in reproducibility.

Enter your paired data scientist, who can raise a hand when it becomes challenging to follow the code, highlight groups of code to break up into pipeline segments and suggest blocks of repeated similar code to bundle into reusable functions. Note that this works bidirectionally: when practicing pairing, the data scientist who is typing is fully aware of the shared nature of code ownership and is proactively driven to make efforts to write reproducible code. Pairing is thus an enabler for creating and maintaining a reproducible data science codebase.

If pair programming is new to your data science practice, we hope this post encourages you to explore it with your team. At Tanzu Labs, we have introduced pair programming to many of our client-side data scientists and have observed that the cycles of continuous communication and feedback inherent in pair programming instill a way of working that sparks more creativity in data discovery, facilitates lean experimentation in model training and promotes better reproducibility of the codebase. And lets not forget that we do all of this to deliver outcomes that delight users and drive meaningful business value.

Here are some practical tips to get started with pair programming in data science:

See the original post here:

Why Data Science Teams Should Be Using Pair Programming - The New Stack

UC Berkeley experts taught ChatGPT how to quickly create datasets on difficult-to-aggregate research about certain materials that can be used to fight climate change, according to a new paper published in the Journal of the American Chemical Society.

These datasets on the synergy of the highly-porous materials known as metal-organic frameworks (MOFs) will inform predictive models. The models will accelerate chemists ability to create or optimize MOFs, including ones that alleviate water scarcity and capture air pollution. All chemists not just coders can build these databases due to the use of AI-fueled chatbots.

In a world where you have sparse data, now you can build large datasets, said Omar Yaghi, the Berkeley chemistry professor who invented MOFs and an author of the study. There are hundreds of thousands of MOFs that have been reported, but nobody has been able to mine that information. Now we can mine it, tabulate it and build large datasets.

This breakthrough by experts at the College of Computing, Data Science, and Societys Bakar Institute of Digital Materials for the Planet (BIDMaP) will lead to efficient and cost-effective MOFs more quickly, an urgent need as theplanetwarms. It can also be applied to other areas of chemistry. It is one example of how AI can augment and democratize scientific research.

We show that ChatGPT can be a very helpful assistant, said Zhiling Zheng, lead author of the study and a chemistry Ph.D. student at Berkeley. Our ultimate goal is to make [research] much easier.

Other authors of the study, ChatGPT Chemistry Assistant for Text Mining and Prediction of MOF Synthesis, include the Department of Chemistrys Oufan Zhang and the Department of Electrical Engineering and Computer Sciencess Christian Borgs and Jennifer Chayes. All are affiliated with BIDMaP, except Zhang.

Certain authors are also affiliated with the Kavli Energy Nanoscience Institute, the Department of Mathematics, the Department of Statistics, the School of Information and KACST-UC Berkeley Center of Excellence for Nanomaterials for Clean Energy Applications.

The team guided ChatGPT to quickly conduct a literature review. They curated 228 relevant papers. Then they enabled ChatGPT to process the relevant sections in those papers and to extract, clean and organize that data.

To help them teach ChatGPT to generate accurate and relevant information, they modified an approach called prompt engineering into ChemPrompt Engineering. They developed prompts that avoided asking ChatGPT for made up or misleading content; laid out detailed directions that explained to the chatbot the context and format for the response; and provided the large language model a template or instructions for extracting data.

The chatbots literature review and the experts approach was successful. ChatGPT finished in a fraction of an hour what would have taken a student years to complete, said Borgs, BIDMaPs director. It mined the synthetic conditions of MOFs with 95% accuracy, Yaghi said.

"AI has transformed many other sectors of our society," said Omar Yaghi, BIDMaP's co-director and chief scientist. "Why not transform science?"

One big area of how you do AI for science is probing literature more effectively. This is really a substantial jump in doing natural language processing in chemistry, said Chayes, dean of the College of Computing, Data Science, and Society. And to use it, you can just be a chemist, not a computer scientist.

This development will speed up MOF-related science work, including those efforts aimed at combating climate change, said Borgs. With natural disasters becoming more severe and frequent, we need that saved time, he said.

Yaghi noted that using AI in this way is still new. Like any new tool, experts will need time to identify its shortcomings and address them. But its worth investing the effort, he said.

If we don't use it, then we can't make it better. If we can't make it better, then we will have missed a whole area that society is already using, Yaghi said. AI has transformed many other sectors of our society commerce, banking, travel. Why not transform science?

Originally posted here:

ChatGPT accelerates chemistry discovery for climate response ... - University of California, Berkeley

In data science, the accuracy of predictive models is vitally important to ensure any costly errors are avoided and that each aspect is working to its optimal level. Once the data has been selected and formatted, the data needs to be cleaned, a crucial stage of the model development process.

In this article, we will provide an overview of the importance of data cleaning in data science, including what it is, the benefits, the data cleaning process, and the commonly used tools.

In data science, data cleaning is the process of identifying incorrect data and fixing the errors so the final dataset is ready to be used. Errors could include duplicate fields, incorrect formatting, incomplete fields, irrelevant or inaccurate data, and corrupted data.

In a data science project, the cleaning stage comes before validation in the data pipeline. In the pipeline, each stage ingests input and creates output, improving the data each step of the way. The benefit of the data pipeline is that each step has a specific purpose and is self-contained, meaning the data is thoroughly checked.

Data seldom arrives in a readily usable form; in fact, it can be confidently stated that data is never flawless. When collected from diverse sources and real-world environments, data is bound to contain numerous errors and adopt different formats. Hence, the significance of data cleaning arises -- to render the data error-free, pertinent, and easily assimilated by models.

When dealing with extensive datasets from multiple sources, errors can occur, including duplication or misclassification. These mistakes greatly affect algorithm accuracy. Notably, data cleaning and organization can consume up to 80% of a data scientist's time, highlighting its critical role in the data pipeline.

Below are three examples of how data cleaning can fix errors within datasets.

Data Formatting

Data formatting involves transforming data into a specific format or modifying the structure of a dataset. Ensuring consistency and a well-structured dataset is crucial to avoid errors during data analysis. Therefore, employing various techniques during the cleaning process is necessary to guarantee accurate data formatting. This may encompass converting categorical data to numerical values and consolidating multiple data sources into a unified dataset.

Empty/ Missing Values

Data cleaning techniques play a crucial role in resolving data issues such as missing or empty values. These techniques involve estimating and filling in gaps in the dataset using relevant information.

For instance, consider the location field. If the field is empty, scientists can populate it with the average location data from the dataset or a similar one. Although not flawless, having the most probable location is preferable to having no location information at all. This approach ensures improved data quality and enhances the overall reliability of the dataset.

Identifying Outliers

Within a dataset, certain data points may lack any substantive connection to others (e.g., in terms of value or behavior). Consequently, during data analysis, these outliers possess the ability to significantly distort results, leading to misguided predictions and flawed decision-making. However, by implementing various data cleaning techniques, it is possible to identify and eliminate these outliers, ultimately ensuring the integrity and relevance of the dataset.

Data cleaning provides a range of benefits that have a significant impact on the accuracy, relevance, usability, and analysis of data.

The data cleaning stage of the data pipeline is made up of eight common steps:

Large datasets that utilize multiple data sources are highly likely to have errors, including duplicates, particularly when new entries haven't undergone quality checks. Duplicate data is redundant and consumes unnecessary storage space, necessitating data cleansing to enhance efficiency. Common instances of duplicate data comprise repetitive email addresses and phone numbers.

To optimize a dataset, it is crucial to remove irrelevant data fields. This will result in faster model processing and enable a more focused approach toward achieving specific goals. During the data cleaning stage, any data that does not align with the scope of the project will be eliminated, retaining only the necessary information required to fulfill the task.

Standardizing text in datasets is crucial for ensuring consistency and facilitating easy analysis. Correcting capitalization is especially important, as it prevents the creation of false categories that could result in messy and confusing data.

When working with CSV data using Python to manipulate it, analysts often rely on Pandas, the go-to data analysis library. However, there are instances where Pandas fall short in processing data types effectively. To guarantee accurate data conversion, analysts employ cleaning techniques. This ensures that the correct data is easily identifiable when applied to real-life projects.

An outlier is a data point that lacks relevance to other points, deviating significantly from the overall context of the dataset. While outliers can occasionally offer intriguing insights, they are typically regarded as errors that should be removed.

Ensuring the effectiveness of a model is crucial, and rectifying errors before the data analysis stage is paramount. Such errors often result from manual data entry without adequate checking procedures. Examples include phone numbers with incorrect digits, email addresses without an "@" symbol, or unpunctuated user feedback.

Datasets can be gathered from various sources written in different languages. However, when using such data for machine translation, evaluation tools typically rely on monolingual Natural Language Processing (NLP) models, which can only handle one language at a time. Thankfully, during the data cleaning phase, AI tools can come to the rescue by converting all the data into a unified language. This ensures greater coherence and compatibility throughout the translation process.

One of the last steps in data cleaning involves addressing missing values. This can be achieved by either removing records that have missing values or employing statistical techniques to fill in the gaps. A comprehensive understanding of the dataset is crucial in making these decisions.

The importance of data cleaning in data science can never be underestimated as it can significantly impact the accuracy and overall success of a data model. With thorough data cleaning, the data analysis stage is likely to output flawed results and incorrect predictions.

Common errors that need to be rectified during the data cleaning stage are duplicate data, missing values, irrelevant data, outliers, and converting multiple data types or languages into a single form.Nahla Davies is a software developer and tech writer. Before devoting her work full time to technical writing, she managed among other intriguing things to serve as a lead programmer at an Inc. 5,000 experiential branding organization whose clients include Samsung, Time Warner, Netflix, and Sony.

Read the original here:

The Importance of Data Cleaning in Data Science - KDnuggets