Workshop on Modern Optimisation Techniques and Deep Learning 

Date: Friday 29th of May 2026

Venue: LT908, Department of Mathematics and Statistics,

Livingstone Tower, 26 Richmond Street, Glasgow, G1 1XH

Poster Submission: send an abstract to ma-contact@strath.ac.uk

Registration Deadline: 20th May 2026 at 5pm

Aims

The proposed workshop targets the increasingly important interface between optimisation and deep learning, with imaging and vision as a central application domain. Modern deep learning models in imaging and vision give rise to massive optimisation problems, often involving millions of variables even for conceptually small tasks. There is growing recognition that further progress will require optimisation methods that go beyond standard gradient-based approaches, motivating the development of more efficient, scalable, and mathematically rigorous algorithms. This also enables a small, focused event in which early-career researchers and PhD students can interact closely with internationally recognised experts. This is an opportunity that is rarely feasible at large conferences.

Organisers

• Prof Ken Chen, 
• Dr Estefania Loayza Romero 

If you require further information, please email ma-contact@strath.ac.uk 

Final Programme

Day 1: Friday 29th of May 2026

University of Strathclyde, LT908, Livingstone Tower, 26 Richmond Street, Glasgow, G1 1XH

 

08:45 - 09:00	Registration and Welcome Coffee
Session 1: Riemannian Optimisation and Optimal Control
09:00 - 09:30	Ya-xiang Yuan  Academy of Mathematics and Systems Science, Chinese Academy of Sciences  Optimization on product manifolds under a preconditioned metric
09:30 - 09:45	Estefania Loayza Romero  University of Strathclyde   Gradient-based training for multilevel optimal control problems with neural surrogate models
09:45 - 10:00	Alex Wray  University of Strathclyde  Optimal control of multiphase continuum systems
Session 2: Nonlinear and Constrained Optimisation
10:05 - 10:35	Yu-Hong Dai  Academy of Mathematics and Systems Science, Chinese Academy of Sciences  A natural SQP method with potentially infeasible subproblems for nonlinear optimization
10:35 - 11:05	Geovani N Grapiglia  Catholic University of Louvain  On the Complexity of Lower-Order Implementations of Higher-Order Methods
11:05 - 11:25	Coffee Break
Session 3: Neural Networks and Deep Learning Theory
11:25 - 11:55	Xin Liu  Academy of Mathematics and Systems Science, Chinese Academy of Sciences  Optimization of Neural Network Wave Functions in Variational Monte Carlo
11:55 - 12:10	Tiffany Vlaar  University of Glasgow  Dissecting Deep Neural Networks
12:10 - 12:40	Andreas Grivas  University of Edinburgh  Detecting unargmaxable outputs in low-rank classifiers
12:40 - 13:25	Lunch Break
Session 4: Deep Learning Applications in Science and Medicine
13:30 - 14:00	Surajit Ray University of Glasgow Tumour Segmentation for Lymphoma Using a Pre-trained U-Net Model and Probability Contour Framework
14:00 - 14:15	Ricky Zhen Qiu University of Strathclyde AI at the Interface of Biomedical Engineering and Neuroscience: Deciphering the Brain Across Scales
14:15 - 14:30	Faxian Cao University of Strathclyde Towards a Digital Twin Workflow for Renal Interventions: An Extensible Multi-Modal Imaging Platform
Session 5: Applied Optimisation in Communications and Manufacturing
14:35 - 15:05	Cong Sun  Beijing University of Posts and Telecommunications  Location optimization of RIS aided broadcast channel
15:05 - 15:20	Abhilash Puthanveettil Madathil  University of Strathclyde  XAI-enabled product/process fingerprints in smart manufacturing
15:20 - 15:40	Coffee Break
Session 6: Variational Methods and Image Processing
15:45 - 16:15	Ken Chen University of Strathclyde Enforcing Diffeomorphic Constraints in Image Registration: Variational and Deep Learning Approaches
16:15 - 16:30	Luke Byrne University of Strathclyde Balancing Complexity and Density: Improving Image Model Generalisation with SinGLU and SFC-Conv
16:30 – 16:45	Hok Shing Wong University of Bath A bilevel learning framework for variational image reconstruction with learned regularizers
16:45	Close

 

ABSTRACTS:

1. Ya-xiang Yuan (yyx@lsec.cc.ac.cn): Optimization on prodect manifolds under a preconditioned metric 

ABSTRACT: Since optimization on Riemannian manifolds relies on the chosen metric, it is appealing to know that how the performance of a Riemannian optimization method varies with different metrics and how to exquisitely construct a metric such that a method can be accelerated. To this end, we propose a general framework for optimization problems on product manifolds endowed with a preconditioned metric, and we develop Riemannian methods under this metric. Generally, the metric is constructed by an operator that aims to approximate the diagonal blocks of the Riemannian Hessian of the cost function. We propose three specific approaches to design the operator: exact block diagonal preconditioning, left and right preconditioning, and Gauss–Newton type preconditioning. Specifically, we tailor new preconditioned metrics and adapt the proposed Riemannian methods to the canonical correlation analysis and the truncated singular value decomposition problems, which provably accelerate the Riemannian methods. Additionally, we adopt the Gauss–Newton type pre-conditioning to solve the tensor ring completion problem. Numerical results among these applications verify that a delicate metric does accelerate the Riemannian optimization methods. Joint work with Bin Gao and Renfeng Peng 

 

2. Xin Liu (liuxin@lsec.cc.ac.cn): Optimization of Neural Network Wave Functions in Variational Monte Carlo  

ABSTRACT: The central challenge in computational materials science and quantum chemistry remains the solution of the electronic Schrödinger equation, a task complicated exponentially by system size. Neural network variational Monte Carlo (NN-VMC) offers a transformative approach, achieving remarkable accuracy with significantly reduced computational expense. Yet, the very flexibility of neural network wavefunctions introduces a critical optimization bottleneck, characterized by high-dimensional, stochastic, and non-convex landscapes. In this talk, I will review the evolution of optimization strategies in NN-VMC, tracing developments from stochastic reconfiguration to modern approximate second-order algorithms and insights from geometric learning. I will highlight the persistent challenges that hinder scalability and efficiency and will outline a path forward. By advancing optimization methods in tandem with neural network design and hardware capabilities, NN-VMC can unlock the study of larger, more complex quantum systems, shifting its role from a tool that follows experiment to one that can direct it. 

 

3. Yu-Hong  Dai (dyh@lsec.cc.ac.cn): A natural SQP method with potentially infeasible subproblems for nonlinear optimization  

ABSTRACT: The sequential quadratic programming (SQP) method has shown remarkable performance for addressing nonlinear optimization problems. However, it typically requires the quadratic programming (QP) subproblems to be feasible. To overcome this limitation, various approaches introducing penalizations or perturbations to the QP subproblems have been developed. In this study, we propose a novel natural SQP algorithm that iterates through a stationary point of the classical QP subproblem, specifically the minimizer closest to the feasible region. This approach ensures global convergence under standard assumptions by identifying a solution with the least constraint violation. The resulting solution minimizes the objective function within the set of minimizers for constraint violation. Furthermore, the proposed method exhibits a quadratic convergence rate. When the original problem is feasible, our assumptions and conclusions align with those of the classical SQP method. Numerical experiments validate the effectiveness and demonstrate the superior performance of the proposed algorithm. This is a joint work with Wenhao Fu. 

 

4. Cong Sun (suncong86@bupt.edu.cn): Location optimization of RIS aided broadcast channel 

ABSTRACT: Reconfigurable intelligent surface (RIS) is a popular technology for 6G. The joint optimization model of RIS location and parameter for RIS aided multi-user broadcast channel is proposed. With highly nonlinear objective function and discrete phase constraints, it is formulated as a mixed integer programming problem. The Courant penalty function technique from optimization and the maximum ratio transmission technique from communication are applied for problem simplification. Then the RIS parameters are updated in closed form, and its location variables are solved through the trust region method. Numerical tests show solid improvement of the quality of service by the new model compared to the work which only optimizes RIS parameters. The advice of RIS location is also provided. 

 

5. Surajit Ray (Surajit.Ray@glasgow.ac.uk): Tumour Segmentation for Lymphoma Using a Pre-trained U-Net Model and Probability Contour Framework 

ABSTRACT: Accurate tumour segmentation is essential for quantitative PET-CT analysis in lymphoma, informing staging, treatment response, and radiotherapy planning. Traditional threshold-based methods or deterministic deep learning models often struggle with tumour heterogeneity, image variability, and limited datasets, and provide little information on segmentation uncertainty. 

We present a framework combining a pre-trained U-Net deep learning model with a probability contour methodology to produce robust, uncertainty-aware tumour delineations. The U-Net generates voxel-wise probability maps of tumour presence, while the probability contour framework translates these into nested tumour boundaries corresponding to different confidence levels. This probabilistic approach allows clinicians to interpret tumour volumes with quantified uncertainty, enhancing reproducibility and decision-making. 

Compared to conventional methods, this framework delivers greater stability, interpretability, and adaptability, supporting downstream applications such as treatment response assessment and dose painting. By integrating deep learning with probabilistic modelling, it offers a clinically meaningful and transparent tool for lymphoma imaging. 

 

6. Hok Shing Wong (hsw43@bath.ac.uk): A bilevel learning framework for variational image reconstruction with learned regularizers 

ABSTRACT: Inverse problems are ubiquitous in imaging sciences, including tasks such as image denoising, image inpainting, and X-ray computed tomography (CT). A classical reconstruction framework is based on variational regularization, where traditional approaches often rely on hand-crafted regularizers such as total variation (TV) and total generalized variation (TGV). With the success of deep learning in image reconstruction, there has been increasing interest in data-driven regularizers, potentially replacing manually designed regularizers with neural networks. In this talk, we explore learned convex regularizers within the variational setting, focusing on convex regularizers parameterized by input convex neural networks (ICNNs). We present a bilevel optimization framework that directly incorporates reconstruction quality into the learning process. In addition, we present a novel approach of solving the variational problem within a primal-dual framework that eliminates the nested nature of a neural network. 

 

7. Geovani N Grapiglia (geovani.grapiglia@uclouvain.be): On the Complexity of Lower-Order Implementations of Higher-Order Methods 

ABSTRACT: In this work, we propose a method for minimizing non-convex functions with Lipschitz continuous $p$th-order derivatives, starting from $p \geq 1$. The method, however, only requires derivative information up to order $(p-1)$, since the $p$th-order derivatives are approximated via finite differences. To ensure oracle efficiency, instead of computing finite-difference approximations at every iteration, we reuse each approximation for $m$ consecutive iterations before recomputing it, with $m \geq 1$ as a key parameter. As a result, we obtain an adaptive method of order $(p-1)$ that requires no more than $\mathcal{O}(\epsilon^{-\frac{p+1}{p}})$ iterations to find an $\epsilon$-approximate stationary point of the objective function and that, for the choice $m=(p-1)n + 1$, where $n$ is the problem dimension, takes no more than $\mathcal{O}(n^{1/p}\epsilon^{-\frac{p+1}{p}})$ oracle calls of order $(p-1)$. This improves previously known bounds for tensor methods with finite-difference approximations in terms of the problem dimension. This is a joint work with Nikita Doikov (Cornell University, USA) 

 

8. Ken Chen (k.chen@strath.ac.uk): Enforcing Diffeomorphic Constraints in Image Registration: Variational and Deep Learning Approaches 

ABSTRACT: In this talk, we address the problem of enforcing diffeomorphic constraints in image registration, with a particular focus on providing guarantees of diffeomorphism in deep learning–based predictions for pairs of input images. We first review how such constraints are successfully imposed in classical variational models. We then discuss the key challenges that arise when moving from successful network training to unreliable or non-diffeomorphic predictions at inference time. Finally, we introduce a new transformation-based approach that resolves these challenges in a mathematically rigorous way, while enabling the use of generative diffusion equations for training with small data. We demonstrate the advantages of our approach in delivering provable guarantees through comparisons with several existing methods. 
Joint work with Junping Wei, Huan Han (WHUT), and Jingqiu Deng and Jianping Zhang (XTU) 

 

9. Estefania Loayza Romero (estefania.loayza-romero@strath.ac.uk): Gradient-based training for multilevel optimal control problems with neural surrogate models  

ABSTRACT: In this talk, we will consider multilevel optimisation framework for optimal actuator and control design in semi-discretised dynamical systems, where the actuator placement is optimised based on the performance of the underlying optimal control problem.  To address the computationally challenging evaluation of the value function for a given actuator realisation, we develop gradient-based neural network surrogates within a supervised learning paradigm. This choice enables the use of efficient consensus-based optimisation methods to determine the optimal actuator design. We validate our approach by solving problems associated with the heat equation with space-dependent diffusion parameter and the viscous Burgers’ equation. 

 

10. Luke Byrne (luke.byrne@strath.ac.uk): Balancing Complexity and Density: Improving Image Model Generalisation with SinGLU and SFC-Conv 

ABSTRACT: In the optimisation of neural networks, the goal is typically to converge to solutions which generalise well to unsampled areas of the problem space. Such solutions have been shown to correlate strongly with wide smooth minima. Therefore, proper optimization is just as dependent on the loss landscape defined by the model and data as on the optimizer itself. In this talk we present two methods for reshaping this landscape: modifying model capacity, and modifying the training data manifold. First, we explore Gated Linear Unit (GLU) layers, which have recently become standard in transformers. By adding multiplication between inputs as a primitive, GLUs increase effective model complexity without increasing parameters. We identify SinGLU, a novel Sin activated GLU function which consistently outperforms the industry-standard SwiGLU in image classification tasks. However, we show that this increased model complexity is only beneficial when the problem space is sampled densely enough to prevent over-fitting. To address this, we introduce SFC-Conv, a novel augmentation technique derived from the spectral analysis of repeated random convolutions. We show that repeated convolution acts as a selective filter in the spatial, frequency, and colour domains. SFC-Conv decouples these effects, allowing precise control over the data manifold, substantially improving domain generalisation in our experiments. Together, these techniques demonstrate how balancing model complexity with data density creates an optimization landscape conducive to robust learning. 

 

11. Andreas Griva (Andreas.Grivas@ed.ac.uk): Detecting unargmaxable outputs in low-rank classifiers 

ABSTRACT: Low-rank classifiers are ubiquitous in modern AI. For example, Large Language Models (LLMs) have vocabularies of hundreds of thousands of tokens, yet their feature activations have far fewer dimensions, making their output layer low-rank. In this talk I will demonstrate a vulnerability of such low-rank classifiers: they can have unargmaxable outputs, i.e. outputs that are impossible to predict irrespective of the input. I will then introduce linear programmes to detect unargmaxable outputs in LLMs and text classification models. 

 

12. Abhilash Puthanveettil Madathil (abhilash.p-m@strath.ac.uk): XAI-enabled product/process fingerprints in smart manufacturing 

ABSTRACT: Next-generation high value-added products increasingly rely on complex freeform geometries, tight tolerances, and enhanced functional performance, yet current smart manufacturing systems lack a quantitative and explainable link between manufacturing process parameters, resulting surface geometry, and final product functionality. This fundamental disconnect forces designers to rely on trial-and-error approaches, limits the ability of digital twins to operate at the machine level in real time, and constrains the realisation of autonomous manufacturing systems. While artificial intelligence has shown promise, prevailing black-box models are data-intensive and computationally expensive, making them unsuitable for high-frequency control and self-optimisation. This work addresses these challenges through an explainable artificial intelligence (XAI)–based process and product fingerprint (FP) approach. The fingerprints are compact, physically meaningful features that capture intrinsic relationships between process parameters, surface geometrical characteristics, and functional performance. Product fingerprints enable quantitative sensing of customer requirements and their translation into optimal surface design parameters, while process fingerprints identify dominant parameters governing product variation, reducing dimensionality and eliminating redundant sensing for real-time digital twins. The FP framework is integrated within a digital twin-driven manufacturing architecture to enable explainable prediction, rapid inference, and closed-loop optimisation. The approach is demonstrated through case studies on laser machining of superhydrophobic surfaces and digital twin-assisted product design for high-fatigue-performance aerospace components. These studies achieve up to 55% dimensionality reduction, predictive accuracy exceeding R² = 0.97, and improvements in computational efficiency compared to conventional artificial intelligence models, alongside up to 10-times improved functional performance enabled by digital twin feedback. 

 

13. Ricky Zhen Qiu (z.qiu@strath.ac.uk): AI at the Interface of Biomedical Engineering and Neuroscience: Deciphering the Brain Across Scales 

ABSTRACT: Understanding how humans learn and behave requires deciphering the brain's complexity at micro and macro scales. In this talk, I present AI and engineering approaches to advance our understanding. At the nanoscale, I introduce our work on synaptome mapping (Science 2020), developing deep learning and generative AI to identify the brain-wide spatial and molecular architecture of billions of individual synapses that interconnect our brain cells. I will show how AI discovered the spatial reprogramming of brain’s molecular connectivity by ageing, disease, behaviour and environment (Neuron 2022, Nat. Comm. 2023). At the macroscale, I demonstrate that AI revealed how the brain’s molecular architecture drives brain functions (PLOS Biology, accepted). I will discuss how these approaches facilitate disease diagnosis (Science, under review), advance brain imaging (IEEE IoT 2025), and improve clinical analytics (IEEE TMI, accepted). 

 

14. Tiffany Vlaar (Tiffany.Vlaar@glasgow.ac.uk):  Dissecting Deep Neural Networks 

ABSTRACT: In this talk I will focus on the role played by individual layers and substructures of deep neural networks. Layer-wise sensitivity to the choice of initialisation and optimiser hyperparameter settings varies. I will show that training neural network layers differently can lead to enhanced generalisation performance and/or reduced computational cost, as well as affect resulting  adversarial robustness. 

 

15. Alex Wray (alexander.wray@strath.ac.uk): Optimal control of multiphase continuum systems 

Multiphase flows are ubiquitous in nature and industry, and controlling them has applications everywhere from carbon sequestration to medical diagnostics. As a consequence, control has seen an ever-increasing focus in the field of fluid dynamics. Unfortunately, inverse problems of this type are typically extremely computationally costly, and most existing studies have focussed on single phase flows which admit convenient bases on which to project suitable reduced dimensional models. Multi-phase studies have typically confined their focus to lubrication-type equations, which have limited applicability. 

Here the more complex problem of free-surface flow down an inclined plane is examined, simulated using the volume-of-fluid open-source solver Basilisk. Control is implemented via a high-fidelity reduced order model using a projection method akin to the Method of Weighted Residuals, and using a Model Predictive Control loop to control the direct numerical simulation with judicious use of the model. Actuation is achieved via imposing a spatiotemporally-varying electric potential on an electrode parallel to the substrate. 

The model is investigated in detail, demonstrating a high degree of accuracy even into the short-wave regime. The control mechanism is shown to be applicable to both uniform and non-uniform target states, and the efficacy of the model predictive control loop is investigated across a wide variety of parameters. 

We also briefly examine the application of similar control techniques to other continuum systems. 

 

16. Faxian Cao (f.cao@strath.ac.uk) Towards a Digital Twin Workflow for Renal Interventions: An Extensible Multi-Modal Imaging Platform 

Modern renal intervention planning increasingly relies on accurate patient-specific anatomy extracted from medical imaging. While deep learning has enabled strong performance for individual segmentation tasks, real-world adoption is often limited by fragmented tooling: models are developed and evaluated in isolation, multi-structure outputs (kidney, tumor, vessels) are not consistently coordinated, and downstream perioperative needs such as visualization, export, and workflow integration require substantial manual effort. We present an extensible, system-integrated platform for renal imaging analysis that unifies multiple segmentation models and clinical-facing utilities within a single workflow. The current implementation supports automated segmentation of kidney, renal tumor, and vasculature from preoperative imaging, followed by standardized post-processing and interactive 3D visualization to facilitate rapid review and downstream planning. Importantly, the platform is designed with modular interfaces for data I/O, inference, post-processing, and visualization, enabling new components to be incorporated with minimal disruption. This provides a practical path to multi-modal expansion beyond CT (e.g., MRI) and to future intraoperative integration, such as registration to intraoperative imaging streams and workflow orchestration during procedures. 

We evaluate the platform on public benchmarks (KiTS2019 and KiTS2023) and additional renal imaging cases from the University of Cambridge School of Clinical Medicine, demonstrating stable multi-structure outputs and coherent 3D reconstructions across heterogeneous cohorts. By bridging model-level performance with an end-to-end, extensible software workflow, this work lays the foundation for scalable deployment of renal imaging AI from preoperative assessment to perioperative and intraoperative support.