Author : Hao Dong, Gatan Frusque, Yue Zhao, Eleni Chatzi, Olga Fink

Abstract : Anomaly detection (AD) is essential in identifying rare and often critical events in complex systems, finding applications in fields such as network intrusion detection, financial fraud detection, and fault detection in infrastructure and industrial systems. While AD is typically treated as an unsupervised learning task due to the high cost of label annotation, it is more practical to assume access to a small set of labeled anomaly samples from domain experts, as is the case for semi-supervised anomaly detection. Semi-supervised and supervised approaches can leverage such labeled data, resulting in improved performance. In this paper, rather than proposing a new semi-supervised or supervised approach for AD, we introduce a novel algorithm for generating additional pseudo-anomalies on the basis of the limited labeled anomalies and a large volume of unlabeled data. This serves as an augmentation to facilitate the detection of new anomalies. Our proposed algorithm, named Nearest Neighbor Gaussian Mixup (NNG-Mix), efficiently integrates information from both labeled and unlabeled data to generate pseudo-anomalies. We compare the performance of this novel algorithm with commonly applied augmentation techniques, such as Mixup and Cutout. We evaluate NNG-Mix by training various existing semi-supervised and supervised anomaly detection algorithms on the original training data along with the generated pseudo-anomalies. Through extensive experiments on 57 benchmark datasets in ADBench, reflecting different data types, we demonstrate that NNG-Mix outperforms other data augmentation methods. It yields significant performance improvements compared to the baselines trained exclusively on the original training data. Notably, NNG-Mix yields up to 16.4%, 8.8%, and 8.0% improvements on Classical, CV, and NLP datasets in ADBench. Our source code will be available at https://github.com/donghao51/NNG-Mix

2.Segment Together: A Versatile Paradigm for Semi-Supervised Medical Image Segmentation (arXiv)

Author : Qingjie Zeng, Yutong Xie, Zilin Lu, Mengkang Lu, Yicheng Wu, Yong Xia

Abstract : Annotation scarcity has become a major obstacle for training powerful deep-learning models for medical image segmentation, restricting their deployment in clinical scenarios. To address it, semi-supervised learning by exploiting abundant unlabeled data is highly desirable to boost the model training. However, most existing works still focus on limited medical tasks and underestimate the potential of learning across diverse tasks and multiple datasets. Therefore, in this paper, we introduce a textbf{Ver}satile textbf{Semi}-supervised framework (VerSemi) to point out a new perspective that integrates various tasks into a unified model with a broad label space, to exploit more unlabeled data for semi-supervised medical image segmentation. Specifically, we introduce a dynamic task-prompted design to segment various targets from different datasets. Next, this unified model is used to identify the foreground regions from all labeled data, to capture cross-dataset semantics. Particularly, we create a synthetic task with a cutmix strategy to augment foreground targets within the expanded label space. To effectively utilize unlabeled data, we introduce a consistency constraint. This involves aligning aggregated predictions from various tasks with those from the synthetic task, further guiding the model in accurately segmenting foreground regions during training. We evaluated our VerSemi model on four public benchmarking datasets. Extensive experiments demonstrated that VerSemi can consistently outperform the second-best method by a large margin (e.g., an average 2.69% Dice gain on four datasets), setting new SOTA performance for semi-supervised medical image segmentation. The code will be released.

View original post here:
Applications of Semi-supervised Learning part1(Machine Learning ... - Medium

Author : Xiangyu Gao, Yaping Sun, Hao Chen, Xiaodong Xu, Shuguang Cui

Abstract : Mobile edge computing (MEC) networks bring computing and storage capabilities closer to edge devices, which reduces latency and improves network performance. However, to further reduce transmission and computation costs while satisfying user-perceived quality of experience, a joint optimization in computing, pushing, and caching is needed. In this paper, we formulate the joint-design problem in MEC networks as an infinite-horizon discounted-cost Markov decision process and solve it using a deep reinforcement learning (DRL)-based framework that enables the dynamic orchestration of computing, pushing, and caching. Through the deep networks embedded in the DRL structure, our framework can implicitly predict user future requests and push or cache the appropriate content to effectively enhance system performance. One issue we encountered when considering three functions collectively is the curse of dimensionality for the action space. To address it, we relaxed the discrete action space into a continuous space and then adopted soft actor-critic learning to solve the optimization problem, followed by utilizing a vector quantization method to obtain the desired discrete action. Additionally, an action correction method was proposed to compress the action space further and accelerate the convergence. Our simulations under the setting of a general single-user, single-server MEC network with dynamic transmission link quality demonstrate that the proposed framework effectively decreases transmission bandwidth and computing cost by proactively pushing data on future demand to users and jointly optimizing the three functions. We also conduct extensive parameter tuning analysis, which shows that our approach outperforms the baselines under various parameter settings.

2.Matrix Factorization for Cache Optimization in Content Delivery Networks (CDN) (arXiv)

Author : Adolf Kamuzora, Wadie Skaf, Ermiyas Birihanu, Jiyan Mahmud, Pter Kiss, Tams Jursonovics, Peter Pogrzeba, Imre Lendk, Tom Horvth

Abstract : Content delivery networks (CDNs) are key components of high throughput, low latency services on the internet. CDN cache servers have limited storage and bandwidth and implement state-of-the-art cache admission and eviction algorithms to select the most popular and relevant content for the customers served. The aim of this study was to utilize state-of-the-art recommender system techniques for predicting ratings for cache content in CDN. Matrix factorization was used in predicting content popularity which is valuable information in content eviction and content admission algorithms run on CDN edge servers. A custom implemented matrix factorization class and MyMediaLite were utilized. The input CDN logs were received from a European telecommunication service provider. We built a matrix factorization model with that data and utilized grid search to tune its hyper-parameters. Experimental results indicate that there is promise about the proposed approaches and we showed that a low root mean square error value can be achieved on the real-life CDN log data

Read more:
Research on Cache Optimization part3(Machine Learning) | by ... - Medium

Once again, this paper offers four novel models for DO prediction. The models are composed of an MLP neural network as the core and the TLBO, SCA, WCA, and EFO as the training algorithms. All models are developed and implemented in the MATLAB 2017 environment.

Proper training of the MLP is dependent on the strategy employed by the algorithm appointed for this task (as described in previous sections for the TLBO, SCA, WCA, and EFO). In this section, this characteristic is discussed in the format of the hybridization results of the MLP.

An MLPNN is considered the basis of the hybrid models. As per Section The MLPNN, this model has three layers. The input layer receives the data and has 3 neurons, one for each of WT, pH, and SC. The output layer has one neuron for releasing the final prediction (i.e., DO). However, the hidden layer can have various numbers of neurons. In this study, a trial-and-error effort was carried out to determine the most proper number. Ten models were tested with 1, 2, , and 10 neurons in the hidden layer and it was observed that 6 gives the best performance. Hence, the final model is structured as 361. With the same logic, the activation function of the output and hidden neurons is respectively selected Pureline (x=y) and Tansig (described in Section Formula presentation) 83.

Next, the training dataset was exposed to the selected MLPNN network. The relationship between the DO and water conditions is established by means of weights and biases within the MLPNN (Fig.4). In this study, the role of tuning theses weighst and biases is assigned to the named metaheuristic algorithms. For this purpose, the MLPNN configuration is first transformed in the form of mathematical equations with adjustable weights and biases (The equations will be shown in Section Formula presentation). Training the MLPNN using metaheuristic algorithms is an iterative effort. Hereupon, the RMSE between the modeled and measured DOs is introduced as the objective function of the TLBO, SCA, WCA, and EFO. This function is used to monitor the optimization benhavior of the algorithms. Since RMSE is an error indicator, the algorithms aim to minimize it over time to improve the quality of the weights and biases. Designating the appropriate number of iterations is another important step. By analyzing the convergence behavior of the algorithms, as well as referring to previous similar studies, 1000 iterations were determined for the TLBO, SCA, and WCA, while the EFO was implemented with 30,000 iterations. The final solution is used to constrcuct the optimized MLPNN. Figure5 illustrates the optimization flowchart.

Optimization flowchart of the models.

Furthermore, each algorithm was implemented with nine swarm sizes (NSWs) to achieve the best model configuration. These tested NSWs were 10, 25, 50, 75, 100, 200, 300, 400, and 500 for the TLBO, SCA, WCA, while 25, 30, 50, 75, 100, 200, 300, 400, and 500 for the EFO84. Collecting the obtained objective functions (i.e., the RMSEs) led to creating a convergence curve for each tested NSWs. Figure6 depicts the convergence curves of the TLBO-MLPNN, SCA-MLPNN, WCA-MLPNN, and EFO-MLPNN.

Optimization curves of the (a) TLBO-MLPNN, (b) SCA-MLPNN, (c) WCA-MLPNN, and (d) EFO-MLPNN.

As is seen, each algorithm has a different method for training the MLPNN. According to the above charts, the TLBO-MLPNN, SCA-MLPNN, WCA-MLPNN, and EFO-MLPNN with respective NSWs of 500, 400, 400, and 50, attained the lowest RMSEs. It means that for each model, the MLPNNs trained by these configurations acquired more promising weights and biases compared to eight other NSWs. Table2 collects the final parameters of each model.

The RMSE of the recognized elite models (i.e., the TLBO-MLPNN, SCA-MLPNN, WCA-MLPNN, and EFO-MLPNN with the NSWs of 500, 400, 400, and 50) was 1.3231, 1.4269, 1.3043, and 1.3210, respectively. These values plus the MAEs of 0.9800, 1.1113, 0.9624, and 0.9783, and the NSEs of 0.7730, 0.7359, 0.7794, and 0.7737 indicate that the MLP has been suitably trained by the proposed algorithms. In order to graphically assess the quality of the results, Fig.7a,c,e, and g are generated to show the agreement between the modeled and measured DOs. The calculated RPs (i.e., 0.8792, 0.8637, 0.8828, and 0.8796) demonstrate a large degree of agreement for all used models. Moreover, the outcome of ({DO}_{{i}_{expected }}- {DO}_{{i}_{predicted}}) is referred to as error for every sample, and the frequency of these values is illustrated in Fig.7b,d,f, and h. These charts show larger frequencies for the error values close to 0; meaning that accurately predicted DOs outnumber those with considerable errors.

The scatterplot and histogram of the errors plotted for the training data of (a and b) TLBO-MLPNN, (c and d) SCA-MLPNN, (e and f) WCA-MLPNN, and (g and h) EFO-MLPNN.

Evaluating the testing accuracies revealed the high competency of all used models in predicting the DO for new values of WT, pH, and SC. In other words, the models could successfully generalize the DO pattern captured by exploring the data belonging to 20142018 to the data of the fifth year. For example, Fig.8 shows the modeled and measured DOs for two different periods including (a) October 01, 2018 to December 01, 2018 and (b) January 01, 2019 to March 01, 2019. It can be seen that, for the first period, the upward DO patterns have been well-followed by all four models. Also, the models have shown high sensitivity to the fluctuations in the DO pattern for the second period.

The real and predicted DO patterns for (a) October 01, 2018 to December 01, 2018 and (b) January 01, 2019 to March 01, 2019.

Figure9a,c,e, and g show the errors obtained for the testing data. The RMSE and MAE of the TLBO-MLPNN, SCA-MLPNN, WCA-MLPNN, and EFO-MLPNN were 1.2980 and 0.9728, 1.4493 and 1.2078, 1.3096 and 0.9915, and 1.2903 and 1.0002, respectively. These values, along with the NSEs of 0.7668, 0.7092, 0.7626, and 0.7695, imply that the models have predicted unseen DOs with a tolerable level of error. Moreover, Fig.9b,d,f, and h present the corresponding scatterplots illustrating the correlation between the modeled and measured DOs in the testing phase. Based on the Rp values of 0.8785, 0.8587, 0.8762, and 0.8815, a very satisfying correlation can be seen for all used models.

The error line and scatterplot plotted for the testing data of (a and b) TLBO-MLPNN, (c and d) SCA-MLPNN, (e and f) WCA-MLPNN, and (g and h) EFO-MLPNN.

To compare the efficiency of the employed models, the most accurate model is first determined by comparing the obtained accuracy indicators, then, a comparison between the optimization time is carried out. Table3 collects all calculated accuracy criteria in this study.

In terms of all all accuracy criteria (i.e., RMSE, MAE, RP, and NSE), the WCA-MLPNN emerged as the most reliable model in the training phase. In other words, the WCA presented the highest quality training of the MLP followed by the EFO, TLBO, and SCA. However, the results of the testing data need more discussion. In this phase, while the EFO-MLPNN achieved the smallest RMSE (1.2903), the largest RP (0.8815), and the largest NSE (0.7695) at the same time, the smallest MAE (0.9728) was obtained for the TLBO-MLPNN. About the SCA-based ensemble, it was shown that this model yields the poorest predictions in both phases.

Additionally, Figs.10 and 11 are also produced to compare the accuracy of the models in the form of boxplot and Taylor Diagram, respectively. The results of these two figures are consistent with the above comparison. They indicate the high accordance between the models outputs and target DOs, and also, they reflect the higher accuracy of the WCA-MLPNN, EFO-MLPNN, and TLBO-MLPNN, compared to the SCA-MLPNN.

Boxplots of the models for comparison.

Taylor diagram of the models for comparison.

In comparison with some previous literature, it can be said that our models have attained a higher accuracy of DO prediction. For instance, in the study by Yang et al.85, three metaheuristic algorithms, namely multi-verse optimizer (MVO), shuffled complex evolution (SCE), and black hole algorithm (BHA) were combined with an MLPNN and the models were applied to the same case study (Klamath River Station). The best training performance was achieved by the MLP-MVO (with respective RMSE, MAE, and RP of 1.3148, 0.9687, and 0.8808), while the best testing performance was achieved by the MLP-SCE (with respective RMSE, MAE, and RP of 1.3085, 1.0122, and 0.8775). As per Table3, it can be inferred that the WCA-MLPNN suggested in this study provides better training results. Also, as far as the testing results are concerned, both WCA-MLPNN and TLBO-MLPNN outperformed all models tested by Yang et al.85. In another study by Kisi et al.42, an ensemble model called BMA was suggested for the same case study, and it achieved training and testing RMSEs of 1.334 and 1.321, respectively (See Table 5 of the cited paper). These error values are higher than the RMSEs of the TLBO-MLPNN, WCA-MLPNN, and EFO-MLPNN in this study. Consequently, these model outperform benchmark conventional models that were tested by Kisi et al.42 (i.e., ELM, CART, ANN, MLR, and ANFIS). With the same logic, the superiority of the suggested hybrid models over some conventional models employed in the previous studies49,65 for different stations on the Klamath River can be inferred. Altogether, these comparisons indicate that this study has achieved considerable improvements in the field of DO prediction.

Table4 denotes the times elapsed for optimizing the MLP by each algorithm. According to this table, the EFO-MLPNN, despite requiring a greater number of iterations (i.e., 30,000 for the EFO vs. 1000 for the TLBO, SCA, and WCA), accomplishes the optimization in a considerably shorter time. In this relation, the times for the TLBO, SCA, and WCA range in [181.3, 12,649.6] s, [88.7, 6095.2] s, and [83.2, 4804.0] s, while those of the EFO were bounded between 277.2 and 296.0s. Another difference between the EFO and other proposed algorithms is related to two initial NSWs. Since NSW of 10 was not a viable value for implementing the EFO, two values of 25 and 30 are alternatively considered.

Based on the above discussion, the TLBO, WCA, and EFO showed higher capability compared to the SCA. Examining the time of the selected configurations of the TLBO-MLPNN, SCA-MLPNN, WCA-MLPNN, and EFO-MLPNN (i.e., 12,649.6, 5295.7, 4733.0, and 292.6s for the NSWs of 500, 400, 400, and 50, respectively) shows that the WCA needs around 37% of the TLBOs time to train the MLP. The EFO, however, provides the fastest training.

Apart from comparisons, the successful prediction carried out by all four hybrid models represents the compatibility of the MLPNN model with metaheuristic science for creating predictive ensembles. The used optimizer algorithms could nicely optimize the relationship between the DO and water conditions (i.e., WT, pH, and SC) in the Klamath River Station. The basic model was a 361 MLPNN containing 24 weights and 7 biases (Fig.4). Therefore, each algorithm provided a solution composed of 31 variables in each iteration. Considering the number of tested NSWs and iterations for each algorithm (i.e., 30,000 iterations of the EFO and 1000 iterations of the WCA, SCA, and TLBO all with nine NSWs), it can be said that the outstanding solution (belonging to the EFO algorithm) has been excerpted among a large number of candidates (=130,0009+310009).

However, concerning the limitations of this work in terms of data and methodology, potential ideas can be raised for future studies. First, it is suggested to update the applied models with the most recent hydrological data, as well as the records of other water quality stations, in order to enhance the generalizability of the models. Moreover, further metaheuristic algorithms can be tested in combination with different basic models such as ANFIS and SVM to conduct comparative studies.

The higher efficiency of the WCA and EFO (in terms of both time and accuracy) was derived in the previous section. Hereupon, the MLPNNs constructed by the optimal responses of these two algorithms are mathematically presented in this section to give two formulas for predicting the DO. Referring to Fig.4, the calculations of the output neuron in the WCA-MLPNN and EFO-MLPNN is expressed by Eqs.5 and 6, respectively.

$$ begin{aligned} DO_{WCA - MLPNN } & = , 0.395328 times O_{HN1 } + 0.193182 + O_{HN2 } - 0.419852 times O_{HN3 } + 0.108298 times O_{HN4 } \ & quad +, 0.686191 times O_{HN5 } + 0.801148 times O_{HN6 } + 0.340617 \ end{aligned} $$

(5)

$$ begin{aligned} DO_{EFO - MLPNN } & = 0.033882 times {{O}_{HN1}}^{prime} - 0.737699 times {{O}_{HN2}}^{prime} - 0.028107 times {{O}_{HN3}}^{prime} - 0.700302 \ & quad times {{O}_{HN4}}^{prime} + 0.955481 times {{O}_{HN5}}^{prime} - 0.757153 times {{O}_{HN6}}^{prime} + 0.935491 \ end{aligned} $$

(6)

In the above relationships, ({O}_{HNi}) and ({{O}_{HNi}}^{prime}) represent the outcome of the ith hidden neuron in the WCA-MLPNN and EFO-MLPNN, respectively. Given Tansig (x) = (frac{2}{1+ {e}^{-2x}}) 1 as the activation function of the hidden neurons, ({O}_{HNi}) and ({{O}_{HNi}}^{prime}) are calculated by the below equations. As is seen, these two parameters are calculated from the inputs of the study, i.e., (WT, pH, and SC).

$$ left[ {begin{array}{*{20}c} {O_{HN1 } } \ {O_{HN2 } } \ {O_{HN3 } } \ {O_{HN4 } } \ {O_{HN5 } } \ {O_{HN6 } } \ end{array} } right] = Tansigleft( {left( {left[ {begin{array}{*{20}c} { - 1.818573} & {1.750088} & { - 0.319002} \ {0.974577} & {0.397608} & { - 2.316006} \ { - 1.722125} & { - 1.012571} & {1.575044} \ {0.000789} & { - 2.532009} & { - 0.246384} \ { - 1.288887} & { - 1.724770} & {1.354887} \ {0.735724} & { - 2.250890} & {0.929506} \ end{array} } right] left[ {begin{array}{*{20}c} {WT} \ {pH} \ {SC} \ end{array} } right] } right) + left[ {begin{array}{*{20}c} {2.543969} \ { - 1.526381} \ {0.508794} \ {0.508794} \ { - 1.526381} \ {2.543969} \ end{array} } right]} right) $$

(7)

$$ left[ {begin{array}{*{20}c} {O_{HN1}{prime} } \ {O_{HN2}{prime} } \ {O_{HN3}{prime} } \ {O_{HN4}{prime} } \ {O_{HN5}{prime} } \ {O_{HN6}{prime} } \ end{array} } right] = Tansigleft( {left( {left[ {begin{array}{*{20}c} {1.323143} & { - 2.172674} & { - 0.023590} \ {1.002364} & {0.785601} & {2.202243} \ {1.705369} & { - 1.245099} & { - 1.418881} \ { - 0.033210} & { - 1.681758} & {1.908498} \ {1.023548} & { - 0.887137} & { - 2.153396} \ {0.325776} & { - 1.818692} & { - 1.748715} \ end{array} } right] left[ {begin{array}{*{20}c} {WT} \ {pH} \ {SC} \ end{array} } right] } right) + left[ {begin{array}{*{20}c} { - 2.543969} \ { - 1.526381} \ { - 0.508794} \ { - 0.508794} \ {1.526381} \ {2.543969} \ end{array} } right]} right) $$

(8)

More clearly, the integration of Eqs.(5 and 7) results in the WCA-MLPNN formula, while the integration of Eqs.(6 and 8) results in the EFO-MLPNN formula. Given the excellent accuracy of these two models and their superiority over some previous models in the literature, either of these two formulas can be used for practical estimations of the DO, especially for solving the water quality issue within the Klamath River.

Continue reading here:
Predicting water quality through daily concentration of dissolved ... - Nature.com

Founded in 2021 by Nicole Baker and Bruno Ohana, Dublin-based Biologit has been helping companies automate the monitoring of scientific literature.

Whether it is human medicines or medical devices, monitoring health products for safety is a very important part of keeping patients safe, says Nicole Baker, an immunologist by background who started her own company to help life sciences firms automate safety monitoring.

Adverse events are one of the top causes of hospitalisation and can lead to serious health issues. Hence, regulators around the world pay very close attention to the surveillance of health products.

Baker, who co-founded Biologit with tech expert Bruno Ohana, specialises in the field of pharmacovigilance, which involves reviewing the vast number of medical extracts published each year and forums on social media to identify any red flags regarding adverse effects of drugs on the market.

This, combined with her experience working in the biotech and pharma industries, helped Baker create Biologit two years ago, leveraging AI to help keep patients safe by simplifying the detection of adverse events from drug development to post-market.

At Biologit, we specialise in cutting-edge active safety surveillance solutions across the life sciences spectrum. We do that by combining expert domain knowledge with the latest technology to build solutions that help keep patients safe, she explains.

The idea is to help companies of all sizes and at any stage of clinical development automate the comprehensive and time-consuming task of monitoring scientific literature.

With its roots in Trinity College Dublin, where the technology was first incubated, Biologit was part of Enterprise Irelands New Frontiers Entrepreneur Development Programme, in which Baker participated in 2019. She then went on to participate inBig Ideas 2020.

The companys first product, Biologit MLM-AI, was developed by iterating with early adopters to build a solution that is fit for the needs of the industry.

From the user experience to the AI, everything was built from the ground up with domain experts. This has given us insights on how to apply AI to solve the problems that mattered most to our users while maintaining high levels of compliance in a regulated industry, Baker explains.

Because we tackled the challenges of the entire safety surveillance workflow, our platform has the unique ability to deliver high levels of automation and productivity gains.

According to her, MLM-AI includes a rich and comprehensive scientific literature database which contains more than 45m citations and is growing every day.

Our users can benefit from the Biologit database out of the box to run their searches, reducing friction and costs.

After launching MLM-AI last year, Baker and Ohana worked on onboarding its first customers. This year, the focus has been more on customer acquisition and growth, Baker says, as the team has increased its presence on industry forums and other online channels.

We continue to onboard new customers and were building a user base that is quite global, Baker says, adding that she has no interest in raising investment at the moment. Weve put a lot of energy in hiring too and have been very fortunate to build a stellar team to help us deliver for our customers and grow Biologit into the future.

Earlier this year, the Dublin-headquartered start-up announced plans to at least double its team in 2023 following a successful 2m funding round led by Enterprise Ireland. At the time, it had 14 employees based across Ireland, India, Poland, France, Spain and the Philippines.

While for Baker the main challenge in running Biologit is finding the time to do the most important tasks, Ohana said there have been a few exciting challenges for the team as a whole.

There were lots of fun challenges since we started, and they have evolved with the different stages of the company: can we build the technology, can we find good market fit and the right business model, will the platform scale as we grow, said Ohana, an expert in machine learning.

It is really nice to get to think about all that, and a great learning experience. Our customers expect a very high standard of security, compliance, and that we continue to innovate for them, were now putting the structure in place to so that we can do those things as we scale.

10 things you need to know direct to your inbox every weekday. Sign up for theDaily Brief, Silicon Republics digest of essential sci-tech news.

Continue reading here:
Biologit: Machine learning and AI to monitor medical literature - SiliconRepublic.com