Category Archives: Data Science

Augmenting the Analyst: Using data science, training, tools, and techniques to enhance performance – Security Boulevard

The increasing demand for cybersecurity analysts is a combination of playing catch-up, keeping up with growing threats/attacker capabilities, and a globally expanding IT footprint. With relief for the growing security skills gap nearly a decade out, we must find ways to support the analysts that are already working to protect us. In this blog, we discuss ways to augment their efforts and maximize their time by overcoming some of the key challenges they face.

Why do we need to augment our analysts?The global cybersecurity landscape is in crisis due to the lack of available skilled talent. A recent U.S. survey by Emsi Burning Glass (now Lightcast) showed that one million cybersecurity professionals are working in the industry, yet there are more than 700,000 open roles to be filled. The situation is similarly critical throughout Europe according to LinkedIn data, which indicates a 22% increase in demand for talent last year alone with no sign of slowing down.

Educational institutions, government efforts, and private training programs are creating new candidates as quickly as possible, but it takes five to ten years to create an experienced L3 security operations center (SOC) analyst. Thats clearly a solution for the future. So, what do we do in the meantime?

What about artificial intelligence, machine learning, and data science?Many people believe that machine learning (ML) and artificial intelligence (AI) are going to replace SOC analysts. But thats not going to happen, at least any time in the next couple of decades.

Yes, we have self-driving cars, and yes, a self-driving car that drives on the road without crashing is impressive. But they are as much enabled by advances in computer vision as they are by AI/ML, Using the same tools to decide if a 10,000-endpoint company network is secure is like keeping 10,000 cars on the road simultaneously when youre not 100 percent sure where youre going or what the road looks like.

AI/ML techniques arent magic bullets to solve the whole problem. They are a collection of solutions to very specific parts of the problem, such as inferring facts about security data that may be difficult or impossible for a human to determine. For example, AI/ML can detect a predictable pattern to user logon failures which highlights it as an automated activity thats using low and slow timing to try and evade detection. Or it can identify anomalous user behavior and connect it to other anomalous system activity such as when an admin suddenly logs onto the system at 3:00am from a new location.

Does the use of AI/ML need any extra training?Data science is a vocation that most security analysts are not skilled or experienced in. AI/ML systems have started to help stem the tide of alerts, but it can become problematic if analysts are not able to understand what these tools are doing.

Early AI/ML tools, for instance, were famous for presenting a result such as anomalous behavior detected, but with no context for the analyst to determine why the behavior was anomalous. The lack of insight had the potential to devolve analysts into a state of environment blindness, allowing critical threats to go unnoticed.

Training provides benefits because security operations center (SOC) analysts want to improve the way they work. Its baked into every modern SOC as the core principle of continual improvement. If we give analysts additional ways to approach the problem space, they will use them to innovate and iterate better ways of creating and delivering security value.

Outside of the data science domain, SOC analysts regularly acquire and keep certifications up to date. But with an expanding number of SOC training courses and certifications available, it is essential that analysts focus on the courses that provide tangible benefits, are relevant to the security domain, and lead to demonstrable improvements in analyst performance and capability.

What tools can help SOC analysts to do more?Modern SOC tools can help make an analyst more effective and productive. These tools take advantage of all types of available security-related data to help analysts perform meaningful analytics. Data is prioritized and presented to the analysts, so they know what to look at first, making it quicker to drill into the important areas.

Similar to AI/ML, automation within SOC tools was cited historically as a way to eliminate the need for analysts. While that debate seems to have ended (for now), some important developments did come out of it.

Specifically, the term Security Orchestration Automation Response (SOAR) has become a key grouping for automated activities. SOAR, though, is much more than this. Its a way to let SOC analysts directly automate the parts of their job which can be automated in a structured, yet collaborative and freeform way with their peers.

For example, SOAR tools can pre-aggregate additional information that an analyst might want to review upon being fed an alert. This is a tremendous time-saver because it cuts out the manual steps of requesting that data.

Click Tax is also a major consideration that doesnt get much attention. This is a colloquial measurement for the time it takes an analyst to interact with and use tools such as loading times, complex chains of UI interactions, the distance of mouse movements, and the potential for errors in selecting or entering data. Click Tax increases the time it takes an analyst to complete a task, and gets in the way of the analysis throughflow. Saving just 30 seconds in Click Tax per alert could save an entire day of SOC analyst time. The title of a recent Forrester report sums it up: Analyst Experience (AX): Security Analysts Finally Escape the Shackles of Bad UX.

ConclusionThe crisis of staffing in the cybersecurity field is going to get worse before it gets better. The good news is that we can help current security analysts be more efficient and effective. We see the best results when cutting-edge technology is used correctly, training is available to help analysts make the best use of it, and tooling is focused on enhancing and augmenting SOC teams to do more better and faster. Combining data science, training, tools, and techniques with great analysts is where the magic happens.

*** This is a Security Bloggers Network syndicated blog from SilverSky authored by michele-johnston. Read the original post at: https://www.silversky.com/blog/augmenting-the-analyst-using-data-science-training-tools-and-techniques-to-enhance-performance/

Continued here:

Augmenting the Analyst: Using data science, training, tools, and techniques to enhance performance - Security Boulevard

Black Knight’s Rapid Analytics Platform transforms the way companies leverage data and enable analytics – HousingWire

The fintech solutions today may have been unimaginable just a few years ago. But between the shift to an online business landscape brought on by the pandemic and increasing demand for digital-first solutions, the mortgage industry was quick to adapt to the new normal of conducting transactions through the virtual world.

Black Knight, a leader in providing innovations that enhance how businesses work with technology, data and analytics, has stayed ahead of the digital mortgage revolution since day one. The companys Rapid Analytics Platform (RAP), for example, launched in 2019 andsimplifies a clients data and analytics workflow. RAP enables users to easily connect to diverse data assets and run advanced analytics on a single platform with real-time, high-speed processing capabilities to deliver fast results even in the most complex cases.

Described as a virtual lab, RAP is being utilized by forward-thinking mortgage, real estate and capital markets professionals for the most advanced capabilities. From portfolio retention strategy, equity analysis and valuation to prepayment and default analytics, pre- and post-bid due diligence, performance benchmarking, and more, RAP significantly reduces the time, effort and cost for organizations in the mortgage industry to analyze large data sets and create models.

Behind RAPs distinctive innovation is Black Knights team of experienced mortgage professionals who knew that clients could benefit significantly by realizing the business value of the companys data and analytics. But in order for clients to glean actionable intelligence from that data about customers, prospects, opportunities and risk, an innovative tool would have to be built first. This led to the creation of RAP a single, cloud-based environment with massive storage, compute and data science capabilities.

Since the launch of RAP, client adoption has accelerated quickly. As a result, Black Knight continues to make valuable platform enhancements, such as increasing scalability and adding new data sets.

The assets available within RAP include nationwide public-records property and MLS information, loan-level and historical mortgage performance data, daily rate lock data, AVMs, home price indices and more. Users have the choice to build their own analytics or leverage Black Knights highly experienced professionals to develop and deliver customized analytics, providing an extra level of available support.

With RAP, clients have a unique tool that arms them with the information needed to better allocate their resources more effectively. From risk management and portfolio research to economic modeling, benchmarking and investor reporting, RAP transforms how companies leverage data and support decision science strategies in their workflow.

By delivering computational power significantly greater than what most organizations have in-house, RAP is democratizing access to big data and analytics allowing mid-sized and smaller lenders to contend in a competitive industry.

With an array of tailored features and a cutting-edge approach, RAP is truly fintech at the forefront of digital mortgage transformation.

View original post here:

Black Knight's Rapid Analytics Platform transforms the way companies leverage data and enable analytics - HousingWire

Introducing the Class of 2026: Selina Manua | BU Today – Boston University

Working for educational equality in Indonesia since sixth grade

Computing and data science major Selina Manua (CDS26) aims to blend her coding skills with her concern for equality in education while at BU. Photo by Cydney Scott

Student ProfileWorking for educational equality in Indonesia since sixth grade

A quick snapshot of the Class of 2026, the highest achieving and most diverse class in BU history: of the 3,620 entering freshmen, 19.4 percent are first-generation students and 21.2 percent are from underrepresented groups. They hail from 47 states (with Mississippi, West Virginia, and North Dakota the exceptions) and 65 countries and regions (23 percent are international students). The class is 59 percent female, 41 percent male, and has an average GPA of 3.9 percent.

But those are just numbers. To give you a better sense of just how accomplished the Class of 2026 is, we introduce four students who share their inspiring stories.

Many students find their involvement in activism and commitment to social justice ratchets up when they get to college. For Selina Manua, growing up in Jakarta, Indonesia, it started in sixth grade.

A family friend shared a news report about a bare-bones one-room school serving the children of a shantytown under a toll bridge. The school was set to be demolished for a public works project, and adults were movingsuccessfully in the endto save it. But Manua (CDS26) asked her mother to take her there, a 30-minute drive from their home, to see the school for herself. When she did, she volunteered to teach math to the children, who were only a few years younger than her, on Saturdays.

But she wanted to do more for the schools 60 students, boys and girls age 6 to 10.

I was shocked about the conditions they were in, Manua says. There was no [insulation] so it was very hot and there was water dripping from the roof, the children had to sit on the floor all the time, and it was poorly lit.

I was in schools that had nice facilities, so I decided to help them have better conditions, because we need to have equality, she says. Thats when I started a group called Reading Train.

Manuas parents, who encourage her giving back, work in real estate development, so she lived in a nicer part of the city and knew people and companies that could help. She set about raising money, collecting in-kind donations, such as furniture, and selling T-shirts. In six months, she collected the equivalent of $10,000, along with in-kind donations to the school, called Sekolah Anak Kolong Tol Jembatan Tiga.

Theres no official translation for the school name, but if it were to be translated, it would be School for Children under the Toll Bridge, Manua says. It sounds sadly Dickensian, but there is a bit of a happy ending.

We raised enough funds to renovate the school and make it a better place for the children and also renovate the small building next to it so they will have, like, a mini-library, she says. We fixed the roof, we painted the walls, we bought new bookshelves, desks and chairs, and insulation and fans, and fixed the floor because the tiles were all broken.

She is still involved with the school, and even wrote and published a reading primer for the students. She continued to teach there on Saturdays, when the students all get a free lunch and are sometimes treated to special events, like a magician.

She wont be there this fall, though. Manua arrived at BU a few days ago and hopes to continue working toward educational equity for women and underserved populations while here. She is a Presidential Scholar and is joining the inaugural class of data science majors in the new Faculty of Computing & Data Sciences.

Reading Train was hardly her only prosocial activity back home though.

Manua attended the same K-12 school through her high school freshman year, and STEM was a major focus there. Thats where my interest really blossomed, she says. There were a lot of math olympiads and competitions, and our teachers really supported us in learning about science and performing experiments.

As a freshman, she studied writing computer code and found her (other) calling. The reason is, coding is more logic-based, and there are a multitude of ways to get the same outcome, she says. It just depends on how you think and how your logic flows.

When she changed schools after freshman year, she formed sheSTEM, an all-female club, because of what she saw among poor girls in the shantytown school and others where she volunteered.

Since theyre raised in a more traditional setting, theyre engineered to think that science or STEM is a scary thing for women, she says. They havent been exposed to the modern thinking that science is for everyone. Thats when I realized we need more inclusivity in STEM, especially for women.

The club, which has about 20 members, organizes events for schools and other nonprofit groups, such as an introduction to the history of women Nobel Prize winners in science and an actual visit to the shantytown school from a woman scientist who conducted an experiment with the kids.

I was in schools that had nice facilities, so I decided to help them have better conditions, because we need to have equality.

Afterward a lot of them said, Oh, this is so fun, especially hearing from the guest talking about what she does as a woman in STEM, Manua says. She hopes the students will realize that STEM shouldnt be for just a single gender, but for everyone in the world.

She strongly believes that a good education is for everyone in the world, and shes set about working to help other students. One of the problems in that school was that the parents are illiterate, she says. Literacy to me is everything, you need it for every job in life. And in the bigger cities, people at least have access to some education, but in rural villages that privilege is taken away. A lot of places do not even have proper schools because of problems like land scarcity and lack of teachers.

So she launched Roda Belajar (in English, Learning Wheels), a plan to create a mobile school that can travel to remote villages with a teacher on board, providing an education that would not otherwise be available. She envisions a classroom inside a bus that will travel to two or three towns a dayyounger kids, she notes, have short attention spansto provide a few hours of lessons, repeating the route daily. Manua has already designed the bus, she says, and is talking to the Ministry of Education about her idea in hopes of getting it up and running next year, when she is home for the summer.

She also put her nascent programming skills to work on the project.

We need data to identify which areas need the mobile school the most, she says. I looked at data from the Ministry of Education on things like student-teacher ratio and found the places where it was the worst. And I made a route using algorithmic optimization with the data to find out where the mobile school should go first and where it should end.

She does have interests outside STEM and education, including playing tennis and cooking Japanese and Korean food. Her older sister attends the University of Washington, and when asked why she chose BU, she points to the multitude of student organizations she can get involved in.

But the most essential reason Manua came here connects to her major: its the new Center for Computing & Data Sciences, rising 19 stories above Commonwealth Avenue, which is set to open in December.

That they dedicated a whole new building to computing and data science really sparked my interest, because thats where my passion lies, she says. By building that building, they really showed that they are ready to invest in the field, and that its going to be one of their main focuses. That looks like the best building at BU.

Read the original here:

Introducing the Class of 2026: Selina Manua | BU Today - Boston University

From a small village in China to MD Anderson: Genomic medicine researcher looks to the future of big data in cancer care – MD Anderson Cancer Center

As an associate professor in Genomic Medicine, Linghua Wang, M.D., Ph.D., studies how normal cells become cancerous and how cancer cells develop resistance to drugs.

She and her lab are working to identify the earliest events during tumor progression from precancerous diseases to discover new biomarkers and targets for the development of effective interception strategies.

The lab is also focused on understanding cellular plasticity, which is how cancer cells adapt to the microenvironment and avoid being attacked by the immune system or cancer treatments.

Cellular plasticity contributes to cancer development, progression and metastasis. If we can better understand this process, we can develop effective treatment strategies to overcome drug resistance, Wang says.

As she looks toward the future, she reflects on the challenges shes overcome to get to where she is today.

A desire for a different path

Growing up in a small village in China, Wang was expected to follow a traditional path for young women to become a housewife and take care of children. In fact, with three younger brothers at home to take care of, she wasnt supposed to advance beyond middle school.

I knew I wanted a different life for myself, Wang says. She worked hard and earned great grades, which caught the attention of her teachers and the school principal, who encouraged Wangs parents to let her continue her education.

Wang did so well in school that she earned scholarships to pay for college, where her love of learning grew. I never had books of my own to read growing up, she says. The first time I saw the library, I couldnt believe there were so many books.

Earning an M.D., then Ph.D.

Wang grew up with the goal of becoming a doctor so she could help people. After medical school, she earned a license to practice ophthalmology. But a few months later, her husband was admitted to a Ph.D. program in Tokyo, Japan.

I didnt want to live apart, but I knew I would have had to start my medical school all over again to be able to practice in Japan, so I decided to move with my husband and find a new career, she says.

For the first few months in Japan, Wang wasnt sure what she wanted to do with her life. But she did know one thing: I didnt want to be just a housewife, so I started looking for a job that would keep me constantly learning. She was hired as a research fellow in a cancer genetics laboratory.

I learned about cancer cells and couldnt wait to learn more, she says. So, she enrolled in a Ph.D. program in cancer genomics at the University of Tokyo, studying pancreatic cancer.

It opened a whole new world for me and fueled my passion, Wang says. I realized that studying the cancer genome can transform cancerdiagnosis and treatment and help cancer patients. After that, I was hooked on cancer genomics and data science.

Making connections at MD Anderson

After earning her Ph.D., Wang and her family moved to Houston in 2012, where she completed her postdoctoral training at Baylor College of Medicine and joined their research faculty.

She wanted to become an independent investigator so that she could build and grow her own lab. In 2016, she was invited to speak at the Annual Human Genome Meeting, where she met Andy Futreal, Ph.D., chair of Genomic Medicine at MD Anderson.

I walked up to him and asked if he had any tenure-track faculty positions, she recalls. I felt so lucky to meet Dr. Futreal, who recruited me to MD Anderson. He is always there whenever I need his support and he provided the platform for me to find my own way to shine.

Wang credits MD Andersons team science approach for her interest in establishing a lab here in 2017. MD Anderson is an exceptional place to work with resources and facilities unlike anywhere else. We have so many talented scientists here, and it is such a wonderful place to collaborate. Working closely as a team, were making meaningful contributions to patient care, she says.

Wangs lab aims to harness the potential of big data to fight cancer. Im thrilled about the future of big data in cancer care and the work were doing in the lab. I want to bring in new researchers who love the work and are just as motivated and ambitious as I am.

Finding a balance between work and home life

With three young kids at home, Wang says being a mother helps her be a better leader. Parenthood has taught me to communicate more effectively, and to be more compassionate with members of my lab, she says.

It also helps her manage her time. I have a very busy schedule, constantly going from one meeting to the next, and with tight deadlines for grants and manuscripts, she says. So, I have to manage my time wisely to make sure I can spend quality time with my family, too.

Outside the lab, she likes to travel with her family and finds that cooking meals for them feeds her creative side. I love testing new recipes and seeing my family enjoy trying something new, she says. Cooking is my mental break, and its nice to make something without having to look at a screen, like I do throughout the workday.

The future of genomic medicine

Wang believes the rise in data science, machine learning and artificial intelligence will advance precision and predictive oncology and accelerate drug development.

We will be able to accurately predict patients response to therapy as well as the risk of recurrence and adverse effects and choose the best possible treatment for patients, she says.

And, perhaps most importantly, by using big data and predictive analytics to determine cancer risk, Wang believes researchers will be able to identify better biomarkers to detect cancer early and develop better prevention strategies to reduce the risk of getting cancer.

I expect to see successful integration of data science and clinical practice in the near future, Wang says.

Request an appointment at MD Anderson online or by calling 1-877-632-6789.

Read the original here:

From a small village in China to MD Anderson: Genomic medicine researcher looks to the future of big data in cancer care - MD Anderson Cancer Center

Friction in Data Analytics Workflows Causing Stress and Burnout – InfoQ.com

Subscribe on: Apple Podcasts Google Podcasts Soundcloud Spotify Overcast Podcast Feed

Shane Hastie: Good day folks. This is Shane Hastie for the InfoQ Engineering Culture podcast. Today I'm sitting down literally across the world from Matthew Scullion. Matthew is the CEO of Matillion who do data stuff in the cloud. Matthew, welcome. Thanks for taking the time to talk to us today.

Matthew Scullion: Shane, it is such a pleasure to be here. Thanks for having us on the podcast. And you're right, data stuff. We should perhaps get into that.

Shane Hastie: I look forward to talking about some of this data stuff. But before we get into that, probably a good starting point is, tell me a little bit about yourself. What's your background? What brought you to where you are today?

Oh gosh, okay. Well, as you said, Shane, Matthew Scullion. I'm CEO and co-founder of a software company called Matillion. I hail from Manchester, UK. So, that's a long way away from you at the moment. It's nice to have the world connected in this way. I've spent my whole career in software, really. I got started very young. I don't know why, but I'm a little embarrassed about this now. I got involved in my first software startup when I was, I think, 17 years old, back in late nineties, on the run-up to the millennium bug and also, importantly, as the internet was just starting to revolutionize business. And I've been working around B2B enterprise infrastructure software ever since.

And then, just over 10 years ago, I was lucky to co-found Matillion. We're an ISV, which means a software company. So, you're right, we do data stuff. We're not a solutions companies, so we don't go in and deliver finished projects for companies. Rather, we make the technologies that customers and solution providers use to deliver data projects. And we founded that company in Manchester in 2011. Just myself, my co-founder Ed Thompson, our CTO at the time, and we were shortly thereafter joined by another co-founder, Peter McCord. Today, the company's international. About 1500 customers around the world, mostly, in revenue terms certainly, large enterprise customers spread across well over 40 different countries, and about 600 Matillioners. Roughly half R and D, building out the platform, and half running the business and looking after our clients and things like that. I'm trying to think if there's anything else at all interesting to say about me, Shane. I am, outside of work, lucky to be surrounded by beautiful ladies, my wife and my two daughters. And so, between those two things, Matillion and my family, that's most of the interesting stuff to say about me, I think.

Shane Hastie: Wonderful. Thank you. So, the reason we got together to talk about this was a survey that you did looking at what's happening in the data employment field, the data job markets, and in the use of business data. So, what were the interesting things that came out of that survey?

Matthew Scullion: Thanks very much for asking about that, Shane. And you're quite right, we did do a survey, and it was a survey of our own customers. We're lucky to have quite a lot of large enterprise customers that use our technology. I mean, there's hundreds of them. Western Union, Sony, Slack, National Grid, Peet's Coffee, Cisco. It's a long list of big companies that use Matillion software to make their data useful. And so, we can talk to those companies, and also ones that aren't yet Matillion customers, about what they've got going on in data, wider in their enterprise architecture, and in fact, with their teams and their employment situations, to make sure we are doing the right things to make their lives better, I suppose. And we had some hypotheses based on our own experience. We have a large R and D team here at Matillion, and we had observations about what's going on in the engineering talent market, of course, but also feedback from our customers and partners about why they use our technology and what they've got going on around data.

Our hypothesis, Shane, and the reason that Matillion exists as a company, really is, as I know, certainly you and probably every listener to this podcast will have noticed, data has become a pretty big deal, right? As we like to say, it's the new commodity, the new oil, and every aspect of how we work, live and play today is being changed, we hope for the better, with the application of data. It's happening now, everywhere, and really quickly. We can talk, if you want, about some of the reasons for that, but let's just bank that for now. You've got this worldwide race to put data to work. And of course, what that means is there's a constraint, or set of constraints, and many of those constraints are around people. Whilst we all like to talk about and think about the things that data can do for us, helping us understand and serve our companies' customers better is one of the reasons why companies put data to work. Streamlining business processes, improving products, increasingly data becoming the products.

All these things are what organizations are trying to do, and we do that with analytics and data visualization, artificial intelligence and machine learning. But what's spoken about a lot less is that before you can do any of that stuff, before you can build the core dashboard that informs an area of the business, what to do next with a high level of fidelity, before you can coach the AI model to help your business become smarter and more efficient, you have to make data useful. You have to refine it, a little bit like iron ore into steel. The world is awash with data, but data doesn't start off useful in its raw state. It's not born in a way where you can put it to work in analytics, AI, or machine learning. You have to refine it. And the world's ability to do that refinement is highly constrained. The ways that we do it are quite primitive and slow. They're the purview of a small number of highly skilled people.

Our thesis was that every organization would like to be able to do more analytics, AI and ML projects, but they have this kink in the hose pipe. There's size nine boots stood on the hose pipe of useful data coming through, and we thought was lightly causing stress and constraint within enterprise data teams. So we did this survey to ask and to say, "Is it true? Do you struggle in this area?" And the answer was very much yes, Shane, and we got some really interesting feedback from that.

Shane Hastie: And what was that feedback?

Matthew Scullion: So, we targeted the survey on a couple of areas. And first of all, we're saying, "Well, look, this part of making data useful in order to unlock AI machine learning and analytics projects. It may well be constrained, but is it a big deal? How much of your time on a use case like that do you spend trying to do that sort of stuff?" And this, really, is the heart of the answer I think. If you're not involved in this space, you might not realize. Typically it's about 60%, according to this and previous survey results. 60% of the work of delivering an analytics, AI and machine learning use case isn't in building the dashboard, isn't in the data scientist defining and coaching the model. Isn't in the fun stuff, therefore, Shane. The stuff that we think about and use. Rather it's in the loading, joining together, refinement and embellishment of the data to take it from its raw material state, buried in source systems into something ready to be used in analytics.

So, any time a company is thinking about delivering a data use case, they have to think about, the majority of the work is going to be in refining the data to make it useful. And so, we then asked for more information about what that was like, and the survey results were pretty clear. 75% of the data teams that we surveyed, at least, reported to us that the ways that they were doing that were slowing them down, mostly because they were either using outdated technology to do that, pre-cloud technology repurposed to a post-cloud world, and that was slowing this down. Or because they were doing it in granular ways. The cloud, I think many of us think it's quite mainstream, and it is, right? It is pretty mainstream. But it's still quite early in this once-in-a-generation tectonic change in the way that we deliver enterprise infrastructure technology. It's still quite early. And normally in technology revolutions, we start off doing things in quite manual ways. We code them at a fairly low level.

So, 75% of data teams believe that the ways that they're doing migration of data, data integration and maintenance, are costing their organizations both time, productivity and money. And that constraint also makes their lives less pleasant personally as they otherwise could be. Around 50% of our user respondents in this survey revealed this unpleasant picture, Shane, to be honest, of constant pressure and stress that comes with dealing with inefficient data integration. To put it simply, the business wants, needs and is asking for more than they're capable of delivering, and that leads to 50% of these people feeding back that they feel under constant pressure and stress, experiencing burnout, and actually, this means that data professionals in such teams are looking for new roles and looking to go to areas with more manageable work-life balances.

So, yeah, it's an interesting correlation between the desire of all organizations, really, to make themselves better using data, the boot on the hose pipe slowing down our ability to doing that, meaning that data professionals are maxed out and unable to keep up the demand. And that, in turn, therefore, leading to stress and difficulty in attracting and retaining talent into teams. Does that all make sense?

Shane Hastie: It does indeed. And, certainly, if I think back to my experience, the projects that were the toughest, it was generally pretty easy to get the software product built, but then to do the data integration or the data conversions as we did so often back then, and making that old data usable again, were very stressful and not fun. That's still the case.

Matthew Scullion: It's still the case and worse to an order of magnitude because we have so many systems now. Separately, we also did a survey, probably need to work on a more interesting way of introducing that term, don't I, Shane? But we talk to our clients all the time. And another data point we have is that in our enterprise customers, our larger businesses so, this is typically businesses with, say, a revenue of 500 million US dollars or above. The average number of systems that they want to get data out of and put it to work in analytics projects, the average number is just north of a thousand different systems. Now, that's not in a single use case, but it is across the organization. And each of those systems, of course, has got dozens or hundreds, in many cases thousands of data elements inside it. You look at a system like SAP, I think it has 80,000 different entities inside, and that would count as one system on my list of a thousand.

And in today's world, even a company like Matillion, we're a 600-person company. We have hundreds of modern SaaS applications that we use, and I'd be fairly willing to bet that we have a couple of ones being created every day. So, the challenge is becoming harder and harder. And at the other side of the equation, the hunger, the need to deliver data projects much, much more acute, as we race to change every aspect of how we work, live and play, for the better, using data. Organizations that can figure out an agile, productive, maintainable way of doing this at pace have a huge competitive advantage. It really is something that can be driven at the engineering and enterprise architecture and IT leadership level, because the decisions that we make there can give the business agility and speed as well as making people's lives better in the way that we do it.

Shane Hastie: Let's drill into this. What are some of the big decisions that organizations need to make at that level to support this, to make using data easier?

Matthew Scullion: Yeah, so I'm very much focused, as we've discussed already, on this part of using data, the making it useful. The refining it from iron ore into steel, before you then turn that steel into a bridge or ship or a building, right? So, in terms of building the dashboards or doing the data science, that's not really my bag. But the bit that we focus on, which is the majority of the work, like I mentioned earlier, is getting the data into one place, the de-normalizing, flattening and joining together of that data. The embellishing it with metrics to make a single version of the truth, and make it useful. And then, the making sure that process happens fast enough, reliably, at scale, and can be maintained over time. That's the bit that I focus in. So, I'm answering your question, Shane, through that lens, and in my belief, at least, to focus on that bit, because it's not the bit that we think about, but it's the majority of the work.

First of all, perhaps it would be useful to talk about how we typically do that today in the cloud, and people have been doing this stuff for 30 years, right? So, what's accelerating the rate at which data is used and needs to be used is the cloud. The cloud's provided this platform where we can, almost at the speed of thought, create limitlessly scalable data platforms and derive competitive advantage that improves the lives of our downstream customers. Once you've created that latent capacity, people want to use it, and therefore you have to use it. So, the number of data projects and the speed at which we can do them today, massively up and to the right because of the cloud. And then, we've spoken already about all the different source systems that have got your iron ore buried in.

So, in the cloud today, people typically do, for the most part, one of two different main ways to make data useful, to do data integration, to refine it from iron ore into steel. So, the first thing that they do, and this is very common in new technology, is that they make data useful in a very engineering-centric way. Great thing about coding, as you and I know well, is that you can do anything in code, right? And so, we do, particularly earlier technology markets. We hand code making data useful. And there's nothing wrong with that, and in some use cases, it's, in fact, the right way to do it. There's a range of different technologies that we can do, we might be doing it in SQL or DBT. We might be doing it using Spark and Pi Spark. We might even be coding in Java or whatever. But we're using engineering skills to do this work. And that's great, because A, we don't need any other software to do it. B, engineers can do anything. It's very precise.

But it does have a couple of major drawbacks when we are faced with the need to innovate with data in every aspect of how we work, live and play. And drawback number one is it's the purview of a small number of people, comparatively, right? Engineering resources in almost every organization are scarce. And particularly in larger organizations, companies with many hundreds or many thousands of team members, the per capita headcount of engineers in a business that's got 10,000 people, most of whom make movies or earth-moving equipment or sell drugs or whatever it is. It's low, right? We're a precious resource, us engineers. And because we've got this huge amount of work to do in data integration, we become a bottleneck.

The second thing is data integration just changes all the time. Any time I've ever seen someone use a dashboard, read a report, they're like, "That's great, and now I have another question." And that means the data integration that supports that data use case immediately needs updating. So, you don't just build something once, it's permanently evolving. And so, at a personal level for the engineer, unless they want to sit there and maintain that data integration program forever, we need to think about that, and it's not a one and done thing. And so, that then causes a problem because we have to ramp new skills onto the project. People don't want to do that forever. They want to move on to different companies, different use cases, and sorry, if they don't, ultimately they'll probably move on to a different company because they're bored. And as an organization, we need the ability to ramp new skills on there, and that's difficult in code, because you've got to go and learn what someone else coded.

So, in the post-cloud world, in this early new mega trend, comparatively speaking, one of the ways that we make data useful is by hand-coding it, in effect. And that's great because we can do it with precision, and engineers can do anything, but the downside is it's the least productive way to do it. It's the purview of a small number of valuable, but scarce people, and it's hard to maintain in the long term. Now, the other way that people do this is that they use data integration technology that solves some of those problems, but that was built for the pre-cloud world. And that's the other side of the coin that people face. They're like, "Okay, well I don't want to code this stuff. I learned this 20 years ago with my on-premise data warehouse and my on-premise data integration technology. I need this stuff to be maintainable. I need a wider audience of people to be able to participate. I'll use my existing enterprise data integration technology, ETL technology, to do that."

That's a great approach, apart from the fact that pre-cloud technology isn't architected to make best use of the modern cloud, public cloud platforms and hyperscalers likes AWS Azure and Google Cloud, nor the modern cloud data platforms like Snowflake, Databricks, Amazon Redshift, Google BigQuery, et al. And so, in that situation, you've gone to all the trouble of buying a Blu-ray player, but you're watching it through a standard definition television, right? You're using the modern underlying technology, but the way you're accessing it is out of date. Architecturally, the way that we do things in the cloud is just different to how we did it with on-premises technology, and therefore it's hard to square that circle.

It's for these two reasons that today, many organizations struggle to make data useful fast enough, and why, in turn, therefore, that they're in this lose-lose situation of the engineers are either stressed out and burnt out and stuck on projects that they want to move on from, or bored because they're doing low-level data enrichment for weeks, months, or years, and not being able to get off it, as the business' insatiable demand for useful data never goes away and they can't keep up. Or, because they're unable to serve the needs of the business and to change every aspect of how we work, live and play with data. Or honestly, Shane, probably both. It's probably both of those things.

So our view, and this is why Matillion exists, is that you can square this circle. You can make data useful with productivity, and the way that you do it is by putting a technology layer in place, specifically designed to talk to these problems. And if that technology layer is going to be successful, we think it needs to have a couple of things that it exhibits. The first one is it needs to solve for this skills problem, and do that by making it essentially easier whilst not dumbing it down, and by making it easier, making a wider audience of people able to participate in making data useful. Now, we do that in Matillion by making our technology low-code, no-code, code optional. Matillion's platform is a visual data integration platform, so you can dive in and visually load, transform, synchronize and orchestrate data.

That low-code, no-code environments can make a single engineer far more productive, but perhaps as, if not more importantly, it means it's not just high-end engineers that can do this work. It can also be done by data professionals, maybe ETL guys, BI people, data scientists. Even tech-savvy business analyst, financiers and marketers. Anyone that understands what a row and a column is can pretty much use technology like Matillion. And the other thing that the low-code, no-code user experience really helps with is managing skills on projects. You can ramp someone onto a project that's already been up and running much more easily, because you can understand what's going on, because it's a diagram. You can drop into something a year after it was last touched and make changes to it much, much more easily because it's low-code, no-code.

Now, the average engineer, Shane, in my experience, often is skeptical about visual 4GL or low-code, no-code engineering, and I understand the reasons why. We've all tried to use these tools before. But, in the case of data, at least, it can be done. It's a technically hard problem, it's one that we've spent the last seven, eight years perfecting, but you can build a visual environment that creates the high-quality push down ELT instruction set to the underlying cloud data platform as well, if not perhaps even better than we could by hand, and certainly far faster. That pure ELT architecture, which means that we get the underlying cloud data platform to do the work of transforming data, giving us scalability and performance in our data integrations. That's really important, and that can be done, and that's certainly what we've done at Matillion.

The other criteria I'll just touch on quickly. The people that suffer with this skills challenge the most are larger businesses. Smaller businesses that are really putting data to work tend to be either technology businesses or technology-enabled businesses, which probably means they're younger and therefore have fewer source systems with data in. A higher percentage of their team are engineering team members. They're more digitally native. And so, the problem's slightly less pronounced for that kind of tech startup style company. But if you're a global 8,000, manufacturing, retail, life sciences, public sector, financial services, whatever type company, then your primary business is doing something else, and this is something that you need to do as a part of it. The problem for you is super acute.

And so, the second criteria that a technology that's going to solve this problem has to have is it has to work well for the enterprise, and that's the other thing that Matillion does. So, we're data integration for the cloud and for the enterprise, and that means that we scale to very large use cases and have all the right security and permissions technology. But it's also things like auditability, maintainability, integration to software development life-cycle management, and code repositories and all that sort of good stuff, so that you can treat data integration in the same way that you treat building software, with proper, agile processes, proper DevOps, or as we call them in the data space, data-ops processes, in use behind the scenes.

So, that's the challenge. And finally, if you don't mind me rounding out on this point, Shane, it's like, we've all lived through this before. Nothing's new in IT. The example I always go back to is one from, I was going to say the beginning of my career. I'd be exaggerating my age slightly there, actually. It's more like it's from the beginning of my life. But the PC revolution is something I always think about. When PCs first came in, the people that used them were enthusiasts and engineers because they arrived in a box of components that you had to solder together. And then, you had to write code to make them do anything. And that's the same with every technology revolution. And that's where we're up to with data today. And then later, visual operating systems, abstracted the backend complexity of the hardware and underlying software, and allowed a wider audience for people to get involved, and then, suddenly, everyone in the world use PCs. And now, we don't really think about PCs anymore. It's just a screen in our pocket or our laptop bag.

That's what will and is happening with data. We've been in the solder it together and write code stage, but we will never be able to keep up with the world's insatiable need and desire to make data useful by doing it that way. We have to get more people into the pass rush, and that's certainly what we and Matillion are trying to do, which suits everyone. It means engineers can focus on the unique problems that only they can solve. It means business people closer to the business problems can self-serve, and in a democratized way, make the data useful that they need to understand their customers better and drive business improvement.

Shane Hastie: Some really interesting stuff in there. Just coming around a little bit, this is the Engineering Culture podcast. In our conversations before we started recording, you mentioned that Matillion has a strong culture, and that you do quite a lot to maintain and support that. What's needed to build and maintain a great culture in an engineering-centric organization?

Matthew Scullion: Thanks for asking about that, Shane, and you're right. People that are unlucky enough to get cornered by me at cocktail parties will know that I like to do nothing more than bang on about culture. It's important to me. I believe that it's important to any organization trying to be high performance and change the world like, certainly, we are here in Matillion. I often say a line when I'm talking to the company, that the most important thing in Matillion, and I have to be careful with this one, because it could be misinterpreted. The most important thing in Matillion, it's not even our product platform, which is so important to us and our customers. It's not our shiny investors. Matillion was lucky to become a unicorn stage company last year, I think we've raised about 300 million bucks in venture capital so far from some of the most prestigious investors in the world, who we value greatly, but they're not the most important thing.

It's not even, Shane, now this is the bit I have to be careful saying, it's not even our customers in a way. We only exist to make the lives of our customers better. But the most important thing at Matillion is our team, because it's our team that make those customers' lives better, that build those products, that attract those investors. The team in any organization is the most important thing, in my opinion. And teams live in a culture. And if that culture's good, then that team will perform better, and ultimately do a better job at delighting its customers, building its products, whatever they do. So, we really believe that at Matillion. We always have, actually. The very first thing that I did on the first day of Matillion, all the way back in January of 2011, which seems like a long, long time ago now, is I wrote down the Matillion values. There's six of them today. I don't think I had six on the first day. I think I embellished the list of it afterwards. But we wrote down the Matillion values, these values being the foundations that this culture sits on top of.

If we talk to engineering culture specifically, I've either been an engineer or been working with or managing engineers my whole career. So, 25 years now, I suppose, managing or being in engineering management. And the things that I think are the case about engineering culture is, first of all, engineering is fundamentally a creative business. We invent new, fun stuff every day. And so, thing number one that you've got to do for engineers is keep it interesting, right? There's got to be interesting, stimulating work to do. This is partly what we heard in that data survey a few minutes ago, right? If you're making data useful through code, might be interesting for the first few days, but for the next five years, maintaining it's not very interesting. It gets boring, stressful, and you churn out the company. You've got to keep engineers stimulated, give them technically interesting problems.

But also, and this next one applies to all parts of the organization. You've got to give them a culture, you've got to give each other a culture, where we can do our best work. Where we're intellectually safe to do our best work. Where we treat each other with integrity and kindness. Where we are all aligned to delivering on shared goals, where we all know what those same shared goals are ,and where we trust each other in a particular way. That particular way of trusting each other, it's trusting that we have the same shared goal, because that means if you say to me, "Hey, Matthew, I think you are approaching this in the wrong way," then I know that you're only saying that to me because you have the same shared goal as I do. And therefore, I'm happy that you're saying it to me. In fact, if you didn't say it to me, you'd be helping me fail.

So, trust in shared goals, the kind of intellectual safety born from respect and integrity. And then, finally, the interest and stimulation. To me, those are all central to providing a resonant culture for perhaps all team members in an organization, but certainly engineers to work in. We think it's a huge competitive advantage to have a strong, healthy culture. We think it's the advantage that's allowed us, in part, but materially so, to be well on the way to building a consequential, generational organization that's making the world's data useful. Yes, as you can tell, it's something I feel very passionate about.

Shane Hastie: Thank you very much. A lot of good stuff there. If people want to continue the conversation, where do they find you?

Matthew Scullion: Well, me personally, you can find me on Twitter, @MatthewScullion. On LinkedIn, just hit Matthew Scullion Matillion, you'll find me on there. Company-wise, please do go ahead and visit us at matillion.com. All our software is very easy to consume. It's all cloud-native, so you can try it out free of charge, click it and launch it in a few minutes, and we'd love to see you there. And Shane, it's been such a pleasure being on the podcast today. Thank you for having me.

Mentioned

.From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Original post:

Friction in Data Analytics Workflows Causing Stress and Burnout - InfoQ.com

What Schools Miss When Theyre Missing Relationship Data – EdSurge

Last month, a new study in Nature revealed a key predictor of economic mobility: connectedness. Specifically, researchers at Opportunity Insights found that relationships with higher-income students dramatically improved low-income students chances of upward mobility in adulthood, even more than traditional success metrics like school quality.

The Opportunity Insights team garnered praise for the sheer size of the data set they built to reach their findings: Their Social Capital Atlas consists of a staggering 21 billion data points on connection, mined from de-identified Facebook data from 72 million users. The analysis also yielded a new species of school-level data, charting the degree of economic connectedness within individual high schools and colleges across the country.

This new research begs a bigger question for education leaders striving for more equitable outcomes: What kinds of relationship data do schools need to understand the trajectories their students are on, and the relationships and resources at their disposal?

Unfortunately, legacy education data systems rarely contain much in the way of relationship data.

Thats not to say schools fly entirely blind. Schools can keep track of which students are paired with what teachers. They can assign advisors or mentors to students who are struggling. They can administer culture and belonging surveys that measure how students and staff experience and perceive their community.

But rosters and climate surveys only get you so far. They lean institution-centric, rather than student-centric. In other words, they rarely reveal the actual relationships and networks at play in students lives. Moreover, they tell schools nothing about students connections with family, friends, coaches, neighbors and the like that make up a young persons actual network, and often contain valuable assets that schools could tap into.

How might schools go about discovering who students know? One obvious strategy to gain a more complete picture of students networks is to ask students themselves.

Often, this takes the form of an activity called relationship mapping, which I describe in greater detail in a new report for the Christensen Institute, Students hidden networks: Relationship mapping as a strategy to build asset-based pathways.

Relationship mapping has low-tech roots. For decades, social workers have created pen-and-paper ecomaps with clients to reveal their social supports and stressors.

Network mapping, ecomapping, relationship mappingit's all the idea of trying to get on paper, Who are the people in your life? said Sarah Schwartz, a clinical psychologist and leading mentoring researcher whom I interviewed. When I do it with young people, I use a blank piece of paper, put their name in the middle and start drawing lines and asking them, Whos in your school? Whos in your community? Whos in your neighborhood? Who are your caregivers friends? Whos in your religious community? explained Schwartz.

This practice has been slow to migrate from paper into the digital realm. Even fairly popular programs like Harvards Making Caring Commons virtual Relationship Mapping Strategy rely on simple spreadsheets.

Pen-and-paper and spreadsheets may suffice for short activities and small programs. But they risk a static approach to relationship data. With better tools, that data could prove both a powerful and dynamic indicator over time. Luckily, a range of entrepreneurs are starting to build tools that could supercharge schools ability to access and store secure data on students networks in ways that help both young people and the institutions that serve them keep track of their connections.

Some tools have emerged from researchers focused on the power of network science to improve outcomes. For example, a new open-source research tool Network Canvas, developed through the Complex Data Collective, streamlines the process of designing network surveys, interviewing subjects, and analyzing and managing social network data.

Another tool built by researchers at Visible Networks Lab (VNL) called PARTNERme uses an interactive interface where kids and parents can draw their social connections, identify who helps them with things they need, and highlight their most pressing needs with the least amount of social support.

The resulting map aims to make invisible networks visible, according to VNLs founder Danielle Varda, a researcher and faculty at University of Colorado Denver School of Public Affairs.

By visualizing these types of things, we make a very complex problem easier to see and therefore more tangible to address, Varda said.

For the past two years, VNL has worked with the Annie E. Casey Foundation to support youth research fellows conducting qualitative research on how the PARTNERme assessment can best detect social supports in young peoples lives.

Other tools are starting to emerge to help young people identify and maintain connections. Palette is a startup focused on fostering more communication across students support networks. The goal, in founder Burck Smiths words, is to better connect and manage the adults that are most influential in a student's success. Palette is still in beta, but will launch a half dozen or so pilot programs this fall in advising, coaching, mentoring and counseling programs.

Other startups are pairing relationship maps with network-building curriculum. My Opportunity Hub (MyOH), an app in development by Edward DeJesus, founder of Social Capital Builders, Inc., nudges young people to keep the connections in their livesteachers, family members and mentorsupdated on their progress, and to build new connections with those in industries they are interested in. The tool goes hand in hand with DeJesuss Foundations in Social Capital Literacy curriculum, which teaches young people about building and mobilizing networks. The app aims to make maintaining connections more manageable. At any given time in the course of Social Capital Builders experiential curriculum, young people are keeping a select five to six individuals, what DeJesus and his team dub Opportunity Guides, up to date on their successes and challenges.

Tools like MyOH demonstrate the potential of pairing relationship-building curriculum with data and visualization tools. Others are starting to take a similar tack. For example, iCouldBe, an online mentoring program and college and career curriculum, is currently building a student-facing connections map where students will be able to visualize their networks on an ongoing basis. (Notably, students served by iCouldBe prefer the term connections to networks). While students make their way through the curriculum, the map will automatically populate any connections with teachers, coaches, and counselors that students identify, and urges students to develop new connections with people they would like to meet.

For iCouldBe, this marks a promising evolution from data-driven mentorship to data-driven network building. We have this enormous database on the backend of the program and use data science tools to really look at how mentees engage in the program. For every single week of the program we see a weekly score based on mentees and mentors engagement," said Kate Schrauth, executive director of iCouldBe. Were going to be looking to take these data science tools and add all of the metrics from the enhanced connections map so that we can understand how mentees are engaging with these broader networks over longer periods of time.

Better tools for assessing and maintaining connectedness offer myriad upsides when it comes to the complex challenges schools are facing this year. First, as researchers like VNLs Danielle Varda have long documented, connectedness and mental health are deeply intertwined. Given concerns about students mental health are top of mind among district leaders, schools would be wise to not just invest in interventions, but data focused on social connectedness.

Second, mapping networks can help create more resilient systems. In the early months of the pandemic, some school districts were lauded as innovative for initiatives that ensured someoneanyonefrom the district reached out to students daily. As Herculean as those efforts were, they were also a reflection of how ill-prepared schools were to leverage and coordinate existing connections in students lives. If more crises upend school as we know it, data on who students know and can turn to offers an invaluable safety net for centralized systems trying to operate under decentralized conditions.

Of course, limited time, financial resources, and network science expertise in schools may hamper adoption of these kinds of tools. Startups hoping to gain a foothold may need to be as much in the business of relationship mapping development as in the business of change management and consulting (which many of the tool providers above offer). Others are betting on adoption first outside of traditional systems. The first step of our strategy toward greater district adoption of PARTNERme is to partner with community-based organizations that provide services to schools to prove the value of using the tool, said Varda of VNLs approach.

But if the recent buzz around economic connectedness is any indication, there's significant interest from schools and the communities that support them in doubling down on the crucial role that relationships play in young peoples lives. Relationships and the resources they can offeroften dubbed social capitaldrive healthy development, learning and access to opportunity. Its time these connections become part and parcel of the data that schools collect to drive and measure their progress.

Read this article:

What Schools Miss When Theyre Missing Relationship Data - EdSurge

NVIDIA and Dell Technologies Deliver New Data Center Solution for Zero-Trust Security and the Era of AI – NVIDIA Blog

Dell PowerEdge Servers Built With NVIDIA DPUs, NVIDIA GPUs and VMware vSphere 8 to Help Enterprises Boost AI Workload Performance and Build Foundation for Zero-Trust Security; Available to Experience Today on NVIDIA LaunchPad

VMware ExploreNVIDIA today announced a new data center solution with Dell Technologies designed for the era of AI, bringing state-of-the-art AI training, AI inference, data processing, data science and zero-trust security capabilities to enterprises worldwide.

The solution combines Dell PowerEdge servers with NVIDIA BlueField DPUs, NVIDIA GPUs and NVIDIA AI Enterprise software, and is optimized for VMware vSphere 8 enterprise workload platform, also announced today.

Enterprises can experience the combination of these technologies on NVIDIA LaunchPad, a hands-on lab program that provides access to hardware and software for end-to-end workflows in AI, data science and more.

AI and zero-trust security are powerful forces driving the worlds enterprises to rearchitect their data centers as computing and networking workloads are skyrocketing, said Manuvir Das, head of Enterprise Computing at NVIDIA. VMware vSphere 8 offloads, accelerates, isolates and better secures data center infrastructure services onto the NVIDIA BlueField DPU, and frees the computing resources to process the intelligence factories of the worlds enterprises.

Dell and NVIDIAs long tradition of collaborating on next-generation GPU-accelerated data centers has already enabled massive breakthroughs, said Travis Vigil, senior vice president, portfolio and product management, Infrastructure Solutions Group, Dell Technologies. Now, through a solution that brings NVIDIAs powerful BlueField DPUs along with NVIDIA GPUs to our PowerEdge server platform, our continued collaboration will offer customers performance and security capabilities to help organizations solve some of the worlds greatest challenges.

Running on BlueField, vSphere 8 supercharges the performance of workloads. By offloading to the DPU, customers can accelerate networking and security services, and save CPU cycles while preserving performance and meeting the throughput and latency needs of modern distributed workloads. The combination increases performance and efficiency, simplifies operations and boosts infrastructure security for data center, edge, cloud and hybrid environments.

Distributed modern applications with AI/ML and analytics are driving the transformation of data center architecture by leveraging accelerators and providing better security as part of the mainstream application infrastructure, said Krish Prasad, senior vice president and general manager, VMware Cloud Platform Business, VMware. Dell PowerEdge servers built on the latest VMware vSphere 8 innovations, and accelerated by NVIDIA BlueField DPUs, provide next-generation performance and efficiency for mission-critical enterprise cloud applications while better protecting enterprises from lateral threats across multi-cloud environments.

NVIDIA AI Enterprise Support for VMware vSphere 8 Coming SoonAs NVIDIA-Certified Systems, the Dell PowerEdge servers will be able to run the NVIDIA and VMware AI-Ready Enterprise Platform, a solution that features the NVIDIA AI Enterprise software suite and VMware vSphere.

A comprehensive, cloud-native suite of AI and data analytics software, NVIDIA AI Enterprise is optimized to enable organizations to use AI on familiar infrastructure. It is certified to deploy anywhere from the enterprise data center to the public cloud and includes global enterprise support to keep AI projects on track.

An upcoming release of NVIDIA AI Enterprise will bring support for new capabilities introduced in VMware vSphere 8, including the ability to support larger multi-GPU workloads, optimize resources and easily manage the GPU lifecycle.

AvailabilityWith NVIDIA LaunchPad, enterprises can get access to a free hands-on lab of VMware vSphere 8 running on NVIDIA BlueField-2 DPUs.

Dell servers with vSphere 8 on NVIDIA BlueField-2 DPU will be available later in the year. NVIDIA AI Enterprise with VMware vSphere is now available and can be experienced on NVIDIA LaunchPad hands-on labs.

NVIDIA CEO Jensen Huang and VMware CEO Raghu Raghuram discussed how the collaboration is driving the next era of computing in a fireside chat at VMware Explore.

View original post here:

NVIDIA and Dell Technologies Deliver New Data Center Solution for Zero-Trust Security and the Era of AI - NVIDIA Blog

10 Top Predictive Analytics Tools to Know – Built In

Running a business means crunching an endless flow of real time data. Sales percentages, ROI growth, customer retention rates these numbers can tell you a lot about where your company is at the present moment. But without organization, they wont be able to tell you where youre headed next.

The process of turning datasets into forecasts and decisions is a science: predictive analytics. A subset of advanced analytics, it is a form of data science that uses current data points to forecast the likelihood of certain events and give company leaders a blueprint to follow. Predictive analytics tools can be used to anticipate the success of future products, reduce customer churn and nip fraud in the bud. Every company from clothing retailers to airplane manufacturers needs to be able to turn data into actionable insights in order to maintain longevity and stay competitive with their peers.

More on Applying Data ScienceWhat Is Data Analysis? Learn How to Derive Key Insights From Your Data.

But making predictions and pulling meaning from a constant stream of digits and statistics isnt something any human can do alone. Luckily for everyone, there are tech tools available that can process even the largest amount of data sets and help leaders make informed decisions about the future of their companies. Below are 10 of the top predictive analytics tools on the market today.

Pricing: Individual plans start at $5,195 per user. Contact site for enterprise pricing

Alteryx is an end-to-end predictive analytics platform that incorporates machine learning principles to help clients easily build forecasting data models. Like other platforms on this list, Alteryx offers collaboration capabilities, but is also built so that users without a coding background can still access insights. The company also offers an analytics process automation platform so that users can unify all their data science and analytics operations in one central location, making monitoring and deployment more straightforward.

Pricing: Custom pricing is available, contact site for details

Adopted by Cisco, Dell, GM and other major companies, Emcien is a full spectrum predictive analytics tool that can integrate with platforms like Tableau and Salesforce to build comprehensive data forecasts. Emcien can turn raw data into business predictions to help customers reduce churn and improve their customer retention initiatives, making it more of an ideal tool for marketing and retail clients. The platform delivers predictions based on real time data and can organize insights into a variety of visualization formats beyond simple graphs, according to its website.

Pricing: Contact site for pricing

Specifically designed for the finance industry, FICO Predictive Analytics offers tools for tracking, modeling and forecasting financial and other relevant data, according to its site. Alongside its predictive analytics dashboard, FICO also offers a decision management platform for companies to manage governance and deal with risks to their datas security. The platforms focus may not make it ideal for customers in all industries, but FICOs services can support clients in healthcare, retail and transportation as well as finance, according to its website.

More on Data Science ToolsThe 7 Best Data-Visualization Tools Experts Recommend

Pricing: Free trial available, contact site for pricing

H2O.ai is a cloud-based predictive analytics tool that uses AI and machine learning technology to help customers build scale data models and forecast future data trends. The platform can handle data prediction types like metric learning, time series forecasting, text classification and regression, according to its site. H2O.ais advantage is its open source model, which makes it a more flexible and scalable solution than other proprietary models. Its AI capabilities can also predict bias in datasets and gives users the ability to control the parameters of their data analysis in case they want to hone in on specific small models.

Pricing: Contact site for pricing

Oracle Data Science is a comprehensive data tracking tool that can be used to organize existing data and transform it into predictive models so companies can make informed strategic decisions. Since Oracle DataScience is included in Oracles product database, users can also access the companys cloud and artificial intelligence tools as needed. In addition to constructing data models, Oracle Data Science users can also store datasets in the cloud for instant accessibility and synchronization across their orgs, according to the companys site.

Pricing: Standard license starts at $2,089 per user annually

Geared toward market researchers, Q Research is a data analytics and forecasting tool that can quickly record and interpret data automatically, according to its site. Datasets can be imported from Q Research to presentation platforms like PowerPoint, and users can view their data forecasts in different formats like predictive trees and cluster tables. Q Researchs existing client base includes household names like Amazon, Meta and Nielsen.

Pricing: Contact site for pricing

RapidMiner is a predictive analytics dashboard that is capable of forecasting, fraud detection, churn prevention and a variety of other data capabilities. Its data science platform gives users access to technology like AI app building, model creation and governance management, according to its site. RapidMiner also provides customers with a variety of plugins like Python scripting, web mining and text processing, and other extensions to amplify their data research.

Pricing: $22 monthly per user

SAP Predictive Analytics is a full spectrum analytics tool that allows users to build scale data models and trace connections between existing data to predict future business directions. The programs modeler tool lets companies look into regression, time series and clustering data models, which can be exported in a variety of formats, according to the companys site. It can also load data from a number of different sources including Microsoft Excel and SQL View. According to a review on peer tech review site G2, SAP Predictive Analytics users may need some technical expertise and prior training in order to take full advantage of the tool. This may make SAP Predictive Analytics less intuitive than other simpler platforms, but its advanced capabilities may make it an ideal solution for companies that need to perform more complicated data analytics tasks.

More on Data Science Applications22 Data Science Applications and Examples

Pricing: Contact site for pricing

SAS Advanced Analytics provides clients with tools to turn large amounts of data into large scale forecasts automatically with the help of AI technology. In addition to its predictive analytics services, SAS Advanced Analytics also offers text mining and data visualization tools to help clients take full advantage of their available data, according to its website. Marketed specifically for enterprise usage, it does feature built-in security features to help protect sensitive enterprise data.

Pricing: Plans starting at $25 monthly per user

TIBCO Statisticas platform is built with collaboration in mind, with workflow options that can be shared across multiple teams for heightened visibility. TIBCO Statistica doesnt offer an open source option, but can be scaled to take on both small and large datasets to generate different types of models, according to its website. The platform also offers IoT focused capabilities that other platforms do not. It also is built with open source functionalities to give clients access to a wider range of analytics functions.

Go here to read the rest:

10 Top Predictive Analytics Tools to Know - Built In

Unify your islands of information with a data fabric – TechNative

Every enterprise is under a mandate to be a data-first company by leveraging their data to compete on analytics.

As a result, analytics, AI, and machine learning (ML) have become core technologies for any data-first journey. However, many of these initiatives are failing.

Im sure you know or are experiencing the challenges responsible for these failures as well as the complexity they bring to every organization:

Exploding data volumes across a variety of types and formats

Rapid importance and growth in streaming data

Capturing data from new and emerging technologies then processing it in real-time, i.e., edge

These challenges are hindering data science and analytic teams because each of these locations has become an island of information that requires negotiation with each site owner before data can be accessed. And these information islands are growing in number making data unification and normalization a time-consuming task.

Find, interpret, and use data from anywhere

The answer is data fabric, an integrated layer (fabric) of data and connecting processes. It accelerates insights by automating ingestion, curation, discovery, preparation, and integration across islands of data. Data fabric is not a new technology but one that has become more important as organizations look to compete using analytics. Here are some things you should look for in a data fabric. A single solution should:

* Reduce the risk and costs associated with large analytic systems by integrating the components needed to access, land, process, and index data securely across multiple physical locations

* Increase productivity of data engineers and analysts by aggregating different types, formats, and systems into a single logical data store

* Simplify data access by connecting multiple clusters and physical locations through a single global namespace

* Reduce platform risk by replacing multiple tools and unique security integrations with a single enterprise security integration

An example of such a solution is the HPE Ezmeral Data Fabric, which enables customers to deploy a hybrid data backbone to provide frictionless access to their global enterprise. This single platform scales to exabyte levels, is optimized for high performance read/writes of both tiny and jumbo objects and increases productivity of data teams with a single logical data plane accessed globally through a single namespace. The built-in ecosystem provides data engineers and analysts with a choice of the most popular tools from the Apache Spark community. Support for a wide range of industry standard APIs enables data to be integrated into other systems, such as Apache Hadoop systems.

Integrate your islands of information

A data fabric replaces the complexity, risk, and cost associated with managing multiple unique tools and security systems with a single security integration that spans across on premises, multiple clouds, and edge. Industry analysts agree that becoming a data-first organization requires data fabric technology to provide a unified data layer to data science teams. It provides data consumers with consistent, governed, and performance-optimized data views, no matter where the data or user are located. The right data fabric can simplify all the capabilities your business needs to compete using analytics.

About the author

Joann Starke is senior product marketing engineer at HPE Ezmeral Software. Joanns domain knowledge and technical expertise have contributed to the development and marketing of cloud, analytics, and automation solutions. She holds a B.S. in marketing and computer science. Currently she is the subject matter expert for HPE Ezmeral Data Fabric and HPE Ezmeral Unified Analytics.

Featured image: Shutterstock

Excerpt from:

Unify your islands of information with a data fabric - TechNative

An 80-year-old Indian is taking one of the toughest examinations in the world. Here’s why – Interesting Engineering

He also appeared for the Joint Entrance Examinationan assessment needed to gain admission into engineering colleges in Indiawhich s considered by many to be the toughest entrance exam in India. In 2021, more than 141,600 students took the exam, and just over 41,860 qualified for the next stage in the entrance process.

But nothing fazed Menon.

"My son, who's an advocate, was interested in Data Processing and wanted to take the exam. He asked me if I would be interested. I was down to the idea - I wanted to challenge myself and upgrade my skills. Additionally, I had the complete support of my family. Unfortunately, my son couldn't [take] the exam, but I did," Menon tellsInteresting Engineering (IE)in an interview.

Menon's home state, Kerala, currently has the highest elderly population (16.5 percent) in India, according to the National Statistical Office (NSO)s Elderly in India 2021 report. A decade from now, it will increase to 20.9 percent. Literacy levels among elderly males and females have improved over time in rural and urban areas, and Kerala has consistently remained the most literate state in India, with a 96.2 percent literacy rate.

Menon isn't alone in his pursuit. Like Menon, former solicitor Archie White decided to complete a degree in fine art at the age of 96. Today, he is believed to be Britain's oldest graduate after he completed the degree. White had retired at 92 and had a stroke six years ago. But nothing could stop him. He told ITV News that age was no barrier.

In 2018, a Forbes article stated that adult learners were the majority of degree seekers in the US. A national survey from Champlain College Online, known for its career-focused adult education, found that 60 percent of US adults aged 23 to 55 without a bachelor's degree have considered returning to school.

However, rising costs and student debts were major hindrances.

While many adults may be returning to school to open a "new chapter" in their careers or start a brand new one, other older students begin a degree in their later years to upgrade their skill set in a fast-changing world or simply to challenge themselves. For the rest, who perhaps couldn't go to college for various reasons when they were younger, getting a degree might be a "lifelong goal" that they finally have time for and are able to afford.

Menon happens to fall into the second category.

To achieve his goals, the octogenarian woke up at 5.30 am each day and studied diligently till 10 pm.

"Student life was the best part of my life. There were no responsibilities; all one had to do was study. And now that I returned to the books, it honestly makes me feel like a student," says Menon.

Menon also tells IE that he didn't have a lot of trouble recollecting the concepts. "I have a few physical difficulties, considering my age, but that's about it," he says.

Menon is right. To qualify for the entrance exam, he had to write 16 papers. He also had to appear for weekly exams on four subjects - mathematics, statistics, data processing, and English and had to score a minimum of 50 percent in all four subjects in order to qualify for the entrance examination. "I cleared them all and became qualified to write the entrance examination, which in itself comprises four papers. I've already cleared two of them [Menon was prepping for the third paper at the time of the interview]," he says.

Realizing that only a few coveted seats are on offer at IIT, Menon decided to stay away from a classroom exam prep course and opted instead for online classes.

"I do not want to be an obstacle for the thousands of youngsters who will be appearing for the exam and hog their seats. So, I decided to solely take up an online course [which will be a three-year course]," he says.

You're probably wondering how Menon learned advanced science and math at his age.

Well, he's actually no stranger to these subjects. When he was younger, Menon had completed a degree in mathematics and spent a year studying for a Masters in Statistics. Later, he completed his post-graduate studies in cryogenic engineering at Syracuse University, New York, with a NASA-sponsored scholarship. Upon receiving his degree, he returned home to spend his career working in India.

A decision he regrets even today.

"It was 1970, and we didn't really have cryogenics experts in India, and I didn't get a chance to pursue [that field]. I left [India] in the hopes of serving my country, but I missed a good opportunity to work [in the US]," he says.

Menon is unsure if he will want to work after acquiring new skills. "I just wanted to study something that I haven't earlier. Also, I believe studying at this age would be an inspiration to many, especially youngsters," he says.

And his inspiration? "My elder brother. He was an ardent student - all I've wanted to do is follow his path," says Menon.

Interestingly, Menon has not retired from his job either. "I'm currently working as a consultant at a firm," he says. Does that mean he has no plans to rest? "None," he responds. "Neither do I receive pensions nor do I get any support. I continue to pay income tax at this age - tax paid is of no benefit to me, and it makes me hurt. Nevertheless, I've not felt the need to take a rest," adds Menon.

Read the original:

An 80-year-old Indian is taking one of the toughest examinations in the world. Here's why - Interesting Engineering