Category Archives: Data Science

Top data preparation challenges and how to overcome them – TechTarget

The rise of self-service BI tools enabled people outside of IT to analyze data and create data visualizations and dashboards on their own. That was terrific when the data was ready for analysis, but it turned out that most of the effort in creating BI applications involved data preparation. It still does -- and numerous challenges complicate the data preparation process.

Increasingly, those challenges are faced by business analysts, data scientists, data engineers and other non-IT users. That's because software vendors have also developed self-service data preparation tools. Those tools enable BI users and data science teams to perform the required data preparation tasks for analytics and data visualization projects. But they don't eliminate data prep's inherent complexities.

In the modern enterprise, an explosion of data is available to analyze and act upon to improve business operations. But the data used in analytics applications is often gathered from various sources, both internal and external. Most likely, it is formatted in different ways and contains errors, typos and other data quality issues. Some of it may be irrelevant to the work at hand.

As a result, the data must be curated to achieve the levels of cleanliness, consistency, completeness, currency and context needed for the planned analytics uses. That makes proper data preparation crucial. Without it, BI and analytics initiatives are unlikely to produce the desired outcomes.

Data preparation has to be done within reasonable limits. As Winston Churchill said, "Perfection is the enemy of progress." The goal is to make the data fit for its intended purpose without getting stuck on analysis paralysis or endlessly striving to create perfect data. But it can't be neglected or left to chance.

To succeed, it's important to understand the challenges that data preparation presents and how to overcome them. Many data preparation challenges could be bundled together under the data quality label, but it's useful to differentiate them into more specific issues to help identify, fix and manage the problems. With that in mind, here are seven challenges to be prepared for.

Data analysts and business users should never be surprised by the state of the data when doing analytics -- or worse, have their decisions be affected by faulty data that they were unaware of. Data profiling, one of the core steps in the data preparation process, should prevent that from happening. But there are different reasons why it may not do so, including the following scenarios:

How to overcome this challenge. Solid data profiling needs to be the starting point in the data preparation process. Data preparation tools can help with that: They include comprehensive data profiling functionality to examine the completeness, cleanliness and consistency of data sets in source systems and then in target ones as part of data curation. Done well, data profiling provides the information needed to identify and address many of the data issues listed in the subsequent challenges.

A common data quality issue is fields or attributes with missing values, such as nulls or blanks, zeros that represent a missing value rather than the number 0, or an entire field missing in a delimited file. The data preparation questions raised by these missing values are whether they indicate that there is an error in the data and, if they do, how should that error be handled. Can a valid value be substituted in? If not, should the record (or row) with the error be deleted, or kept but flagged to show there's an error?

If they aren't addressed, missing values and other forms of incomplete data may adversely affect business decisions driven by analytics applications that use the data. They can also cause data load processes that aren't designed to handle such occurrences to fail. That often results in a scramble to figure out what went wrong and undermines confidence in the data preparation process itself.

How to overcome this challenge. First, you need to do data profiling to identify data that's missing or incomplete. Then determine what should be done based on the planned use case for the data and implement the agreed-upon error handling processes, a task that can also be done with a data preparation tool.

Invalid values are another common data quality issue. They include misspellings, other typos, duplicate entries and outliers, such as wrong dates or numbers that aren't reasonable given the data's context. These errors can be created even in modern enterprise applications with data validation features and then end up in curated data sets.

If the number of invalid values in a data set is small, they may not have a significant impact on analytics applications. But more frequent errors may result in faulty analysis of the data.

How to overcome this challenge. The tasks to find and fix invalid data are similar to the ones for handling missing values: Profile the data, determine what to do when errors are encountered and then implement functions to address them. In addition, data profiling should be done on an ongoing basis to identify new errors. This is a data preparation challenge where perfection is not likely to be attained -- some errors will inevitably slip through, but the intent should be to do whatever it takes to keep them from adversely affecting analytics-driven decisions.

One more data quality issue that complicates data preparation is inconsistency in the names and addresses of people, businesses and places. This type of inconsistency involves legitimate variations of that data, not misspellings or missing values. But if not caught when preparing the data, such inconsistencies can prevent BI and analytics users from getting a complete view of customers, suppliers and other entities.

Examples of name and address inconsistencies include the following:

How to overcome this challenge. The source data schemas must be examined to determine what name and address fields are included, and then the data profiled to identify the scope of the inconsistencies. Once you've done that, the following are the three optimal ways to standardize the data:

Inconsistent data also is often encountered when multiple data sources are needed for analytics. In this instance, the data may be correct within each source system, but the inconsistency becomes a problem when data from different sources is combined. It's a pervasive challenge for the people who do data preparation, especially in large enterprises.

How to overcome this challenge. When the data inconsistency is the result of an attribute such as an ID field having different data types or values in different systems, data conversions or cross-reference mapping can be used for a relatively easy fix. However, when it occurs because business rules or data definitions are different across the source systems, analysis must be done to determine data transformations that can be implemented while preparing the data.

One of the key steps in creating the business context needed for analytics is enriching data. Examples of data enrichment measures include the following:

But enriching data isn't an easy task. Deciding what needs to be done in a data set is often complicated, and the required data enrichment work can be a time-consuming procedure.

How to overcome this challenge. Data enrichment should start with a strong understanding of the business needs and goals for analytics applications. That will make it easier to identify the business metrics, KPIs, augmented data and other enrichments required to meet those needs, and then to define things like filters, business rules and calculations to generate the enriched data.

Although data scientists and other analysts perform many ad hoc tasks, the more impactful data preparation work they do inevitably becomes a recurring process that then expands in scope as the resulting analytics becomes more and more valuable. But organizations often encounter problems with that, especially if they're using custom-coded data preparation methods.

For example, what happens and why in a data preparation process is typically known only by the person who created it if there's no documentation of the process or of data lineage and where data is used. The dependency on such individuals requires them to spend increasingly more time on these processes and makes it hard to sustain the data preparation work when they leave the organization.

In addition, when changes or enhancements to a data preparation process are needed, bolting on new code makes the process more precarious and difficult to maintain.

How to overcome this challenge. Data preparation tools can help you avoid these traps and achieve long-term, sustained success in preparing data. They provide productivity and maintenance benefits such as pre-built connectors to data sources, collaboration capabilities, data lineage and where-used tracking and automated documentation, often with graphical workflows.

To succeed at data preparation, it's imperative that you first understand what data is needed for an analytics application and the associated business context. Once the relevant data has been gathered from source systems, the key steps in preparing it include the following:

As you go through those steps, do what's appropriate and possible in a reasonable way, especially in cleansing the data. Keep in mind that perfection often isn't attainable or may not be worth the cost to achieve -- and that it really can be the enemy of progress on data preparation.

Read this article:

Top data preparation challenges and how to overcome them - TechTarget

Make 2022 the year to pursue a data analyst career with this $35 course bundle – ZDNet


The most important decisions that companies make are based on hard data, which is why data analytics skills are highly sought after in the tech industry. So if you're hoping to switch to a career that's stable and in-demand,The 2022 Ultimate Microsoft Excel & Power BI Certification Bundle has the training you need.

Entry-level data scientists need to get acquainted with Microsoft Excel. If you're already familiar with it, the one-hour "Excel Pro Tips: Data Visualization" course can take you way past regular charts into the program's powerful data visualization tools. Then, you can dive deeper into four hours of lessons with "Microsoft Excel: Data Visualization with Charts & Graphs."

Similarly, data analysis requires a firm foundation in both statistics and probability theory. "Mathematics for Data Science" can teach you plenty in just an hour, while "Statistics & Mathematics for Data Science and Data Analytics" provides a comprehensive 11 hours of instruction.

Onto the tools that data scientists use on a daily basis: Microsoft's Power BI is a favorite business intelligence platform, and "Microsoft Power BI: The Complete Master Class" will teach it to you from scratch, focusing on the most important components. A little more experience would be helpful, but not required, for "Up & Running with Power BI Desktop," a thorough hands-on guide. Students were very satisfied with both courses, rating them each 4.6 out of 5 stars.

"Data Visualization with R" is a step-by-step guide that will teach you the fundamentals of the analyst-favorite platform R. Instructor Juan Galvan, founder of Seattle-based Sezmi SEO, shares his expertise that has allowed him to successfully create and sell many products on a variety of online marketplaces.

You will qualify for one of tech's most in-demand positions after taking the "Fullstack Web Development Master Class 2022." It will quickly teach you how to build apps and websites from scratch using cutting-edge NodeJS and MongoDB.

These courses can be accessed on your mobile devices as well as your laptop, so you can study anywhere. But if keeping all of your devices charged becomes a struggle, get the inexpensive compact cable that can charge them all -- even your laptop.

Don't pass up this chance. Right now, The 2022 Ultimate Microsoft Excel & Power BI Certification Bundle is on sale for only $34.99 -- under $4 per course.

Prices subject to change.


Make 2022 the year to pursue a data analyst career with this $35 course bundle - ZDNet

University Lecturer in Demography and Data Science job with UNIVERSITY OF HELSINKI | 278749 – Times Higher Education (THE)

The University of Helsinki is the oldest and largest institution of academic education in Finland, an international scientific community of 40,000 students and researchers. In international university rankings, the University of Helsinki typically ranks among the top 100. The University of Helsinki seeks solutions for global challenges and creates new ways of thinking for the best of humanity. Applicants of any gender, linguistic and cultural background or from members of minority groups are welcomed.

The Faculty of Social Sciences is Finlands leading research and education institute in the social sciences, and the countrys most diverse social sciences faculty in terms of the range of its disciplines and fields of research. The Faculty has a strong international research profile and an international masters programme, and several of its disciplinary units have been rated among the top 50 in the world. The Faculty has approximately 500 research and teaching staff, and each year it awards some 350 bachelors degrees, 400 masters degrees and more than 40 doctoral degrees. For more information, please see the Faculty website at

The Faculty of Social Sciences and the Centre for Social Data Science invite applications for the position of


for a fixed term 5-year appointment beginning as soon as possible, but no later than August 2022.

The duties of the present university lecturer will include providing teaching in accordance with the degree requirements, supervising and examining theses and dissertations, conducting research that is relevant for understanding the causes and consequences of changing family dynamics, and coordinating the activities of the Family Formation in Flux research team that is based at the University of Helsinki ( In addition to the Flux project, the successful candidate will be affiliated with the Centre for Social Data Science ( and with the Population Research Unit (


The appointee shall hold an applicable doctoral degree, for example in demography, sociology, economics, data science, or statistics, and the ability to provide high-quality teaching based on research and to supervise theses and dissertations. The degree requirement must be met by the end of the application deadline.

To successfully attend to the duties of the position, the appointee must have good English skills.

When assessing the qualifications of each applicant, attention will be paid to:

In the evaluation, special emphasis will be put on (1) research and publication track record in the field of demography, and (2) the ability to teach advanced quantitative methods in the social sciences. While part of the work of the successful candidate will include coordinating the activities of the Family Formation in Flux -project, and while we expect research contributions that are relevant for understanding changing family dynamics, at the application and selection stage we value equally existing research track records in social, health, and family demography.


We are an equal opportunity employer and offer an attractive and diverse workplace in an inspiring environment. The annual gross salary range will be approx. 44,00065,500, depending on the appointees qualifications and experience. In addition, University of Helsinki offers comprehensive benefits to its employees, including occupational health care, opportunities for professional development, support for applying for research project funding as well as library, wellbeing and fitness services. Further information is available at The employment contract will include a probationary period of six months.

The chosen applicant is expected to reside in Finland while employed by the University of Helsinki. The Faculty of Social Sciences provides assistance in relocation.


Please submit your application, together with the required attachments, through the University of Helsinki Recruitment System via the link Apply for the position. Applicants who are currently employed by the University of Helsinki are requested to submit their application via the SAP Fiori portal at The closing date for applications is March 6, 2022.

Applicants are requested to enclose with their applications the following documents in English:

Detailed information on the CV and list of publications as well as the presentation of teaching skills can be obtained online:

The enclosures must be submitted as a SINGLE PDF file named as follows: fieldofposition_lastname_firstname. Other enclosures, such as recommendation letters or certificates, are not required.


Further information on the position and the work environment may be obtained from Professor Mikko Myrskyl,

Further information on the recruitment process may be obtained from HR Specialist Minna Maunula,

Technical support for the recruitment system:

Due date

06.03.2022 23:59 EET

Read more here:

University Lecturer in Demography and Data Science job with UNIVERSITY OF HELSINKI | 278749 - Times Higher Education (THE)

Here’s how to navigate ‘The Great Relearning’ revolution | World Economic Forum – World Economic Forum

When COVID-19 forced the brakes on the global economy, millions of people decided to step off the treadmill to refuel their aspirations. As many as 40-75% of the workforce is reported to be considering quitting their current job. This movement has precipitated a talent crisis, fuelling debate on whether this is the Great Resignation or the Great Reshuffle. While each analysis is insightful, I believe we are looking at something completely different; we are at the tipping point of the Great Relearning Revolution.

While attrition numbers have been widely reported, the number of people choosing to learn has not. Enrollment on popular MOOC (massive open online course) platforms has skyrocketed. At Coursera, it was 640% higher from mid-March to mid-April 2020 than during the same period in 2019, growing from 1.6 to 10.3 million. At Udemy, enrolment was up over 400% between February and March 2020. The e-learning market, growing at a compound annual rate of 20%, is on course to reach a trillion dollars by 2027. Among the courses in high demand are data science, artificial intelligence and machine learning. For those struggling to find talent in these areas, thats promising news.

This hunger to relearn within the workforce also reflected some interesting dimensions in other recent surveys. A Gallup-Amazon study revealed that 48% of workers in the US are willing to switch to a new job if offered skills training opportunities and 65% of them believe employer-provided upskilling is very important when evaluating a potential new job. A MetLife survey highlighted an even more interesting insight: two in three (63%) women who left the workforce during the pandemic said they are ready to return and eight in 10 of those are considering careers in science, technology, engineering, and mathematics (STEM).

We seem to be witnessing a redefining of literacy, akin to Alvin Tofflers prophecy, The illiterate of the 21st century will not be those who cannot read and write, but those who cannot learn, unlearn and relearn.

What should companies do in the eye of this storm? Every sailor knows there are only two ways to weather a storm find the nearest port and anchor till it blows over, or adjust its sail, change course and ride the waves. Whats needed in the corporate environment is an adjustment of the sails. Denying this trend, playing it passively, or even reacting too aggressively (think inflated compensation tactics) is perhaps not the best solution, what is needed are three simple but foundational shifts in organizational human resource strategy:

Its time to broaden the employment perspective away from CTCs (cost to company). According to a BCG survey, 68% of workers around the world blue and white-collar alike are willing to retrain and learn new skills. We knew this about Gen Z and the Millennials. Not so well known was that fact that nearly two-thirds of people over 45 are prepared to spend a significant amount of time learning new skills. Interestingly, the perceived value of training and development is reported to have almost doubled in the last five years. Rewarding and actively encouraging this effort could be an effective new lever for reframing the organizational view of talent retention.

Historically, hiring has been anchored in conventional educational qualifications. We are now witnessing the stirrings of a promising new trend in the transition from an employers market to an employees market skill-based hiring. In the US, LinkedIn reports a 21% increase in job postings advertising skills and responsibilities instead of qualifications. But there is still an overriding bias towards qualifications in hiring we need to correct this urgently. Not only is finding a job the ultimate reward for an individuals investment in learning new skills, the right skill-fit results in better performance and a win-win.

Were all aware of the fact that an organizations success is enabled by its people. Its time to turn this philosophy upside down recognizing the fact that organizational success emerges when we become an enabler, when we create pathways to success for our people. A curious headline caught my attention the other day: Why a jungle gym is better than a corporate ladder. The article quoted talent experts advocating lateral moves, as well as dedicated time and money for learning in every company's reskilling plan. Be it offering reskilling opportunities, or providing time-off and budgets for self-directed learning, companies that respond to the growing hunger for learning will find themselves propelled forward by the momentum.

The Fourth Industrial Revolution calls for a new mindset of continual learning. Yet, its very basis internet of things, data analytics, cybersecurity, artificial intelligence and machine learning are all areas of talent scarcity today. Industry 4.0 needs an equally powerful revolution to build a strong foundation and fuel its growth: the Great Relearning Revolution. The key to hope and the lever to success. Ignoring it today would be our loss.

Written by

C. Vijayakumar, Chief Executive Officer and Managing Director, HCL Technologies

The views expressed in this article are those of the author alone and not the World Economic Forum.

Read this article:

Here's how to navigate 'The Great Relearning' revolution | World Economic Forum - World Economic Forum

We are India’s 1st telco to build Big Data AI/ML Cloud Advanced Analytics Platform on AWS: Vodafone Idea –

New Delhi: The Indian telecom industry has been at the heart of the technology driven disruption over the last decade. With over 986 million active wireless subscribers, the industry has an opportunity to also drive transformation for businesses across industries.

IANS spoke to Dr Sanjeev Chaube, EVP and Head, Big Data & Advanced Analytics at Vodafone Idea to get his insights on how Cloud has been at the core of their digital transformation and how they are leveraging intelligent technologies such as artificial intelligence (AI), machine learning (ML), Internet of Things (IoT) and analytics.

Dr Chaube believes that there is a critical need for telecom service providers to transform from 'Telco to Techno' and technologies such as 5G, Artificial Intelligence, Big Data Analytics, Cloud Computing, IoT / IIoT / AIoT, and Robotic Process Automation (RPA) will enable this digital transformation.

Q: What are the key strategic pillars of digital transformation at Indian Telcos as you look to transform from a connectivity provider to an orchestrator of value-added solutions that meet the needs of digitally-savvy, connected customers?

A: I believe businesses are fast turning to digital solutions to empower remote workforces, provide customers with better services, and create an immersive experience with increased visibility, resilience & agility. Moreover the need for eliminating traditional, hardware driven, and large expensive physical operations has been at the core of Digital Transformation worldwide across industries.

We need to understand that any transformation starts with data. Hence, a company captures customer data efficiently and leverages it to make smarter decisions. Goal should be to streamline key customer touchpoints to increase spends & reduce efforts.

The employees need to be well informed with relevant data, data driven decisioning culture, access to learning & development platforms empowering better organisational transformation.

Businesses need to shift from IT departments to consolidated enterprise platforms that can absorb & integrate latest technologies instantly. Datasets, processes, web and apps should be shared across the company and the data needs to be integrated to provide a single source of truth by building appropriate Data Lake, Data Warehouses/Marts over Cloud for massive storage & processing capacities.

However, the choice of technology stack whether Open-Source or Proprietary depends primarily on the Value vs Cost proposition. Also, the need for scope, scale, speed, quality & ease form the basis for finalising the tech stack.

Ensuring near Real-time Customer 360 view in form of Business Intelligence Reports and Dashboard for leadership and at different levels of management ensure better & contextual decision making.

Therefore, in my opinion, the key pillars and ingredients for effective Intelligent Digital Transformation Program are a) Artificial Intelligence/Machine Learning & Data Science for Data Driven Decision Making, b) Customer Relevance & Centricity Business Processes, c) Engage & Empower Employees aligned around customer, d) Digital First Strategy with leanest possible technology stack, e) Bring Speed, Scale, Simplicity & Values with Platform, f) Enhance Personalization and g) Single Source of Truth & Data Engineering

Q: How are you using Big Data, Advanced Analytics, Artificial Intelligence/Machine learning & Data Science to understand and cater to the needs of your customers better?

A: Big Data and Advanced Analytics is extensively adopted to improve customers experiences and business performance. Artificial Intelligence & Data Science techniques especially Machine/Deep Learning algorithms are significantly improving some of the crucial areas/services including Customer Segmentation, Targeted marketing, Personalised offerings & recommendations, Churn Prediction, Product development, Predictive Analytics, Call Drop Analysis, reducing fraud, Price optimization, Network experience optimization, Location based services etc.

Over the next five years, rise in mobile-phone penetration and decline in data costs will add 500 million new Internet users in India, creating opportunities for new businesses. Legacy data sets were all structured in nature, however over the last few years, with the explosion of data, unstructured data is gaining equal or more importance in the market. With the advent of 5G technology, the available data from voice, video, social media, messaging, IOT and all new 5G use case deployments are going to increase multifold from hereon.

This Big Data generated by telco's has all the features like massive volume, variety, velocity & veracity. It therefore becomes very critical for us to first organise and systematically store the data emerging from multiple applications within Data Ocean, Datalake, Data Warehouse /Data Marts be it using Cloudera Hadoop Ecosystem or Cloud infrastructure.

Secondly, this Big Data is then used to perform Data Analytics for decision making at different organisational levels from operations and strategy making perspective. All 5 stages of Data Science and Advanced Analytics lifecycle namely Descriptive, Diagnostic, Predictive, Prescriptive and Pre-Emptive Analytics are performed to extract information and patterns.

Machine learning use cases have great potential & value such as assisting with Customer Acquisition, Retention, Digital Engagement, Anomaly Detection, Root Cause Analysis, Predictive Maintenance, Ticket Classification, SLA Assurance and building intelligent networks with features like Self-Healing, Dynamic Optimization & Automated Network Designs. However, in order to work effectively, they require specific computational, pipeline and support infrastructure as well to support massive data & their parallel processing.

Predictive models whether Real-time or Batch processing basis business requirements leveraging structured data feeds such as demographic, usage, billing etc. or Unstructured data feeds like Chat, Text, Images, Video feeds are built to support business with insights beforehand.

Social media analytics is performed using Computer Vision, Natural Language Processing (NLP) & Text parsing techniques etc. to extract sentiments of customers across the country for decision making. Different Deep Learning Architectures of Neural networks ranging from Convolutional Neural Network, Recurrent Neural Networks, Self-Organizing Maps, Auto encoders, etc. are used appropriately as per desired intent.

Q: What did cloud technology allow you to do that you couldn't do before?

A: Cloud is a key enabler of Large-Scale Transformation: be it benefitting Consumer Analytics, Digital Adoption, and Network Monitoring & Management and bringing efficiency in Operations by facilitating faster decisions at scale and speed.

As on date, we are India's first Telco to build a full-fledged Big Data AI/ML Cloud Advanced Analytics Platform on AWS Cloud.

Before embarking on our cloud journey, our data sets were collected in silos across the country. Then, they were aggregated to run the various machine learning & deep learning algorithms.

Also, as the data sets were huge in volume, the available compute & GPU support for data consolidation in the legacy infrastructure was a challenge. All of these complexities added delays to our operations & hampered the productivity of the teams. With the new data science platform deployed over AWS, we are able to address all of these challenges & achieve much more.

Using the compute & memory power available on cloud, we can consolidate data & effectively run our machine / deep learning algorithms & also do parallel data processing with ease. The time to run these algorithms have already reduced by over 60 per cent, and with further optimization, we expect our data science platform to deliver more in the near future.

Additionally, availability of deep learning frameworks like Tensor flow, MXNet, Keras, Pytorch, Gluon, etc. along with pre-trained ML models as APIs help us experiment more with data sets, at scale & speed & eventually in turn improve our customer experience. This feature rich data science platform with its scalability, elasticity & pay-as-you-go commercial model helps consolidate operations & effectively manage costs.

In the future, we'll be able to integrate & effectively use this platform to address our challenges across new technology initiatives including 5G, Edge, IoT monetization, customer 360, Network Function Virtualization transition & so on.

Q: What are your views on the rapid worldwide progress in AIoT & IIoT Analytics for Connected Intelligence with Cloud Technologies?A: As per Fortune Business Insights, global IoT market size is projected to grow from USD 381.30 billion in 2021 to USD 1854.76 billion in 2028. While if we follow Researchandmarkets report, globally the AIoT market itself will reach $78.3 Billion by 2026. Also, the Global IIoT market which was valued at about $216.13 in 2020 and is expected to grow to about $1.1 trillion by 2028.

The above data clearly sets the roadmap and vision for telecom as a connectivity service provider not only for India but telco's globally.

The use of AI for decision making in IoT and data analytics will be crucial for efficient and effective smart city solutions in terms of decision making. This is in line with the Indian government's plan to develop 100 smart city projects including smart airports, smart railway stations, smart public healthcare services etc., where the 5G, AI, IoT & Cloud technologies will play a vital role in developing these cities & services.

The convergence of AI and Internet of Things (IoT) technologies and solutions (AIoT) is leading to "thinking" networks and systems. The AI-enabled edge device market will be the fastest-growing segment within the AIoT. AIoT automates data processing systems, converting raw IoT data into useful information.

AIoT solutions are the precursor to next-generation AI Decision as a Service (AIDaaS). With AIoT, AI is embedded into infrastructure components, such as programs, chipsets and edge computing, all interconnected with IoT networks. APIs are then used to extend interoperability between components at the device level, software level and platform level. These units will focus primarily on optimising system and network operations as well as extracting value from data.

These solutions will provide customer's ability to acquire leading wireless enabled AIoT products and reduce their time to market.

The key areas of AIoT solutions include Data Services, Predictive Asset Management, Intelligent Immersive Experience, Process Improvement, Next-Gen UI & UX and Intelligent Industrial Automation.

This Intelligent connectivity, Big Data & AI/ML technologies are expected to play a major role in key industrial areas such as Agricultural Productivity, Automotive, Transportation & logistics, Industrial & manufacturing operations, Healthcare, Public safety, and security.

Leveraged with AWS IoT Cloud services to collect, organise and analyse data from industrial & publicly installed sensors & devices at scale and for near real-time decision making, AIoT enables industries to seamlessly manage and control actions across the edge and cloud.

The key components for Artificial Intelligence + IoT (AIoT) Analytics consist of Sensors, IoT Gateway, Network, Storage & Analytics Engine for Data Analysis, Monitoring and Intelligent Automation for actions & notifications at the Edge.

Artificial Intelligence & Machine learning algorithms are leverage both at Cloud & Edge over emerging data from IoT sensors such as Visual/Optical Sensors, Temperature Sensors, Pressure Sensors, Infrared Sensors, Gas, Gyroscope Sensors, Accelerometers, Humidity Sensors, Level Sensors etc. to derive descriptive & predictive insights for effective & real-time monitoring, notifications and action fulfilment.

Continued here:

We are India's 1st telco to build Big Data AI/ML Cloud Advanced Analytics Platform on AWS: Vodafone Idea -

Discover free technology and data science courses at FGV –

a Getulio Vargas Foundation (FGV) offers over a hundred free short courses, in the most diverse fields of knowledge, on its online platform. Among these learning programs, some focus on technology and data science.

When a score equal to or greater than 7 (seven) is obtained, the system creates a statement allowing proof of participation in the course. Programs are self-taught, so that they can be implemented at the most appropriate time for each students reality.

To get started, all you need to do is register on the site. Below, see the free courses:

Credit: StartupStock Images/PixabayFree FGV Data Science and Technology Courses

the chapter Install, customize, and understand core R functions Displays the installation of R and its supporting software. The course also addresses their distinguishing features, in relation to point-and-click programs, characteristic of the Microsoft Windows operating system.

for 60 hours Data Science It brings examples of real applications of knowledge extraction and generation via data science, as well as discussing ethical aspects related to this new field of knowledge.

the chapter Introduction to organizing data for multivariate analysis. It presents the importance of organizing data in multivariate analyzes and lists the practices used in organizing and preparing data for analysis, with the goal of extracting as much information as possible about the available data and ensuring that analyzes are performed with robustness and productivity.

Introduction to complex networks: metrics for centralization. Networks address how they exist in our daily lives, in a variety of situations. In addition, the basic concepts of networks or graphs are introduced, as well as their definition and methods of representation, measurements made at vertices or edges and, finally, classic algorithms that allow for basic analysis of graphs.

the chapter Introduction to factor and data set analysis techniques. It introduces a method of multivariate analysis, called factor analysis, which allows determining the behavior of groups of variables or consumers. The course also covers combined analysis, a tool that allows to predict the reaction of consumers to certain characteristics of products and services, which makes them of great importance in the field of management.

Topics in machine learning Introduces the student to the field of machine learning, introduces related learning methods and concepts. In the course, the student will develop a broad view of machine learning, knowledge of SVM (Support Vector Machines) and an understanding of unsupervised learning methods.

With the current demand for full-time interconnection, the Information Technology (IT) area is gaining increasing importance in the corporate world. Fundamentals of IT management It presents and discusses the role of technology in business, and provides a broad view of how IT operations affect company results.

The field of Information Technology (IT) has gained increasing importance in the corporate world with the current demand for full-time interconnectedness. the chapter The impact of information technology on business It introduces you to the concepts for understanding how technology affects traditional models of competitiveness advocated from the perspectives of differentiation, cost and focus, developed by Michael Porter.

the chapter Internet overexposure It examines the problem and, through hypothetical and real cases, reveals some prevention practices and guidelines on how to act in circumstances in which overexposure actually occurred.

Data for Good Explains how to contribute to the use of information for the benefit of society. The course deals with capabilities and tools aimed at seeking solutions to everyday problems.

FGV has a variety of free courses in different fields of activity due to the partnership with OEG Open Education Global. As a member since 2008, the institution was the first Brazilian to join the consortium of educational institutions from countries offering free online content and educational materials.

See the article here:

Discover free technology and data science courses at FGV -

COVID-related hospitalizations are rising in S.F. Data from UCSF sheds light on how many are for COVID or with COVID – San Francisco Chronicle

COVID-19-related hospitalizations in San Francisco have reached an all-time high. On Tuesday, the city reported 262 COVID-positive patients, three more than the maximum reached during last winters surge. Since then, hospitalizations have continued to rise. By Thursday, there were 274 COVID-related hospitalizations in the city, according to data from the California Department of Public Health.

Along with this surge, an important distinction has become key to understanding COVID-related hospitalizations: Are patients being admitted for COVID or with COVID?

According to data from the UCSF, as of last Wednesday, about 60% of COVID-positive patients across its hospitals were admitted primarily for the virus. The remaining roughly 40% were so-called with-COVID hospitalizations patients admitted for non-COVID ailments but incidentally tested positive for the virus.

While UCSFs for-COVID hospitalizations are closing in on their peak, they remain below what they were last winter. On the other hand, incidental cases are at an all-time high. Before omicron, fewer than 15 daily hospitalizations were categorized as incidental. But so far this month, the health system has averaged almost 40 daily incidental hospitalizations and recorded the highest yet of 51 last Wednesday.

Though the distinction between for-COVID and with-COVID patients only recently became central to understanding COVID trends, UCSF has always been tracking this data, said Rhiannon Croci, a UCSF clinical informatics specialist. But it wasnt until this recent surge of incidental cases that UCSF began reporting on these metrics.

Delineating between the two types of hospitalizations was not particularly valuable during previous surges, when 85-90% of their patients were hospitalized primarily for COVID, said Croci.

Compared to previous stages of the pandemic, primary-COVID patients are less likely to require critical care. During the delta surge last summer, around 40% to 50% of for-COVID hospitalizations at UCSF were in the ICU, but so far this month, that share has stayed below 25%.

The small number of people in the ICU is likely the result of vaccines and boosters. Data from earlier in the month shows that though around 20 people with up-to-date vaccines were in the hospital for COVID, very few ended up in the ICU.

Other health systems and government officials also publish data on incidental COVID hospitalizations. But while most places use patients diagnoses as the primary criterion for determining these cases, UCSF considers another key variable: whether patients receive a 5-day course of remdesivir, an antiviral drug used to treat moderate-to-severe COVID infections.

According to an in-depth analysis conducted by a team of UCSF data scientists, using just the admit diagnosis excluded many patients which, after manual review, were clearly suffering from moderate-to-severe COVID. By incorporating the medication into their definition, the percentage of for-COVID hospitalizations increased from about 50% to roughly two thirds. The share of incidental cases decreased from about 50% to roughly a third, which reflected what they were anecdotally experiencing in the hospital.

Parallel trends were observed at the UC San Diego health system, which uses a similar schema to identify for-COVID hospitalizations. According to Brian Clay who is a chief medical information officer at UCSD, previous waves had between 20% to 25% of incidental cases, but starting in December, that share increased to about 35%. In the past week, hes seen the number rise closer to 50%.

Incidental cases, though less severe, can still have large implications for hospitals. Because these patients can transmit the virus, resource-intensive isolation procedures are still necessary. And for patients, though their COVID infections may be milder, other ailments may be worsened by the virus, still putting them in critical care.

Sara Murray, an associate chief medical information officer and director of the health informatics data science and innovation team at UCSF, believes its important to get an accurate count of which hospitalizations are incidental versus primarily for COVID. Without it, hospitals are unable to decide on system-wide policies, which, at health systems as large as UCSF and UCSD, affect tens of thousands of people.

And as more health systems across the country publish data on incidental cases, having a standard definition is increasingly important, says Murray.

We all need to speak the same language on this, and then we can really understand where we are headed in this pandemic, wrote Murray, calling for the Centers for Disease Control and Prevention to create a standard definition.

Nami Sumida is a San Francisco Chronicle data visualization developer. Email: Twitter: @namisumida


COVID-related hospitalizations are rising in S.F. Data from UCSF sheds light on how many are for COVID or with COVID - San Francisco Chronicle

New year, new skills: Transform your career with upGrad – CNA

Cognisant of this trajectory, Temasek-backed edtech platform upGrad is poised to enrich professionals looking to fast-track their careers in an evolving job market through the acquisition of relevant skills.

Mr Liviu Nedef, upGrads vice president and head of marketing, Asia Pacific, said that todays dynamic economic environment has led to a need for working professionals to develop new skills in order to access roles in the tech, data and digital domains.

Some of upGrads most popular courses are those in innovative technologies such as data science and AI, both of which are critical to Southeast Asias economic transformation. Industries as diverse as entertainment, education, banking and manufacturing are tapping the power of data science to drive growth, while the use of AI has been ramping up throughout the region.

According to Mr Nedef, data science-related career pathways have become very popular in the last five years. Data has become the fuel that powers all types of businesses across industries, helping organisations better understand their customers, facilitating decision-making throughout companies and driving business strategy forward. Data science enables companies to efficiently generate and leverage data from multiple sources to derive valuable insights that inform the next set of management actions, he explained.

For professionals who want to learn new data analytics skills and lay a solid foundation for their career, upGrad offers a comprehensive nine-month fully online Professional Certificate in Data Science and Business Analytics from University of Maryland, a top 100 global university; and a six-month online Data Analytics Certificate from Caltech, a renowned university ranked sixth in the QS Global World Rankings 2022.

Those who wish to deepen their experience and knowledge in data science may benefit from pursuing the 18-month fully online Master of Science in Data Science from Liverpool John Moores University ranked among the top 1 per cent of global universities, or the University of Arizonas MSc in Data Science. The latter counts as one of the top 100 universities globally.

Also from Liverpool John Moores University is the Master of Science in Machine Learning and AI, which gives students the opportunity to work on more than 15 industry projects, multiple programming tools and a dissertation, along with the acquisition of natural language processing, reinforcement learning and deep learning skills.

Additionally, learners will gain full alumni status from the universities upon graduation.


Read more:

New year, new skills: Transform your career with upGrad - CNA

Fugro N : partners with University of Houston on workforce development effort –

The project, "Data Science for the Energy Transition," is being funded through a 3-year grant with the National Science Foundation (NSF) and will offer undergraduate and master's students specialised training in statistical and machine learning techniques for subsurface Geo-data. Fugro's role as an industry partner on the project is to provide UH with real-world Geo-data and guidance on their use for hands-on training opportunities.

Advances in Geo-data science are needed to keep pace with the global demands for renewable energy sources, including offshore wind. Requiring extensive Geodata coverage over vast lease areas, innovative computing techniques can help operators shorten the development schedule by making critical information available more quickly. As an example, Fugro has developed a machine learning solution for mapping boulder fields from seafloor data to uniquely identify and analyse thousands of boulders. Accelerating the site investigation phase through this kind of automation helps lower capital investment and the levelized cost of energy for offshore wind projects.

"We are pleased to partner with UH on this project and are committed to advancing Geo-data analytics and computing skills in the energy sector," said Jason Smith, Fugro's Global Director for Geodata Analysis and Geoconsulting. "Conventional and renewable energy development benefits from more automated application of Geo-data. As a UH alumnus, I am proud to be leading Fugro's involvement on this project and look forward to the partnership's contribution toward a safe and liveable world ."


Fugro NV published this content on 24 January 2022 and is solely responsible for the information contained therein. Distributed by Public, unedited and unaltered, on 24 January 2022 14:03:06 UTC.

View original post here:

Fugro N : partners with University of Houston on workforce development effort -

Data Science is Helping the World Recover from Covid Loss – Analytics Insight

Data science has been improving in many ways in order to help recover the world from covid loss

Data science has become a powerful weapon that is helping the world to recover from covid losses. COVID forced companies to make a full model jump to match the dramatic shift in daily life. Models had to be rapidly retrained and redeployed to try to make sense of a world that changed overnight. Many organizations ran into a wall in data collection but others were able to create new data science processes that could be put into production much faster and easier than what they had before. From this perspective, data science processes have become more flexible. On the flip side, there is another segment of the population that experienced (and continues to experience) economic difficulties as a result of the pandemic. This skews economics, as millions of people are attempting to climb back up to the standard of where they were pre-COVID. People who previously would have played a sizable role in economic models are effectively removed from the equation for the time being.

Data science, in principle, is a very powerful source of technologies that help each of the businesses. Data scientists are big data wranglers, gathering and analysing large sets of structured and unstructured data.

Data science technology, on a basic level, is an extremely strong wellspring of advances that help every one of the organizations. Information researchers are enormous information wranglers, assembling and examining huge arrangements of organized and unstructured information.

Various Approaches to Recover World from Covid Losses:

1. Convey a computerized operational hub: Digital operational hubs that go about as a basic connection between digitalized tasks, cycles, and resources, momentary functional proficiency, and long-haul system have turned into a vital ability during COVID-19. They permit organizations to assemble assets, for example, new information sources and investigation frameworks, to empower business groups to examine arising patterns all the more rapidly, abbreviate input cycles, and acquire knowledge into potential results.

2. Embrace constant information: Monitoring ongoing information from sites, online media, clickstreams, and portable applications has become progressively significant lately. A pioneer no longer has the advantage of sitting tight days and weeks for the most recent data. Different advancements, including informing stages and stream-handling capacities, empower continuous information handling and investigation; the utilization of the half and half cloud permits chiefs to react in hours rather than days or weeks.

3. Focus on social moves: The pandemic showed numerous pioneers that their associations could be more dexterous than they understood they were during an emergency. A developing number of interdisciplinary groups, nimble working techniques, and information-driven mentalities have grown, for the time being, making profoundly designated and productive investigation capacities. Keeping the force going will require developing these movements , for example, reskilling laborers. Such work is as yet conceivable while representatives work from a distance. As a feature of its groundwork for the future, one monetary administrations organization utilized Zoom video preparing to show senior leaders AI ideas, ways of utilizing the innovation, and ways to execute change. Associations can be more precise and quicker at anticipating the changing requirements of their client networks by having an assorted labour force.

4. Embrace an agreeable plan: Analytical advancement: groups can upgrade hazard the board and identification with different exercises and devices, permitting them to incorporate basic oversight into the interaction. For instance, reported rules, agendas, and preparing materials are accessible to set up assorted groups, use hazard measurements, and keep steady over changes, like changes in approaches, laws, and guidelines. Exercises remember putting for place techniques and information apparatuses for distinguishing and moderating danger in information and observing models.

There is no ideal opportunity for carelessness or sentimentality in this new world. What was once ordinary cant be re-established; neither danger nor opportunity is little in this new time. To manage consistent vulnerability, interruption, and steadily evolving conditions, pioneers should get ready associations to flourish in this new climate.

Share This ArticleDo the sharing thingy

About AuthorMore info about author

Analytics Insight is an influential platform dedicated to insights, trends, and opinions from the world of data-driven technologies. It monitors developments, recognition, and achievements made by Artificial Intelligence, Big Data and Analytics companies across the globe.

See original here:

Data Science is Helping the World Recover from Covid Loss - Analytics Insight