Key Skills

Job Description

There are many standard models and methodologies available that can provide the usual point predictions when it comes to time series forecasting, ranging from classical (linear) models to machine learning models to foundation models. However, when not only point predictions are desired, but also a confidence or uncertainty range, then finding a suitable solution becomes much more difficult. Many standard models, such as the bulk of machine learning models, do not provide a way to learn and predict confidence intervals out of the box. Classical models often do provide confidence levels but are generally limited in the types of times series that they can describe well. And the recent foundation models for time series forecasting, created on the success of Large Language Models, may come with the ability to forecast confidence levels, but are still immature and largely unproven. At ING Wholesale Banking Advanced Analytics we research possibilities to send account managers alerts when their clients have exceptionally high or low account balances. For this purpose, we do not so much need to predict the balance itself (point predictions) but rather predict an interval in which the balance is expected to lie. So that an alert is warranted when the balance exceeds this interval. In previous research we focused primarily on (machine learning) models with an inherent capability to produce intervals. We like to continue the research, where we are also interested in methods that may produce intervals for already trained point prediction models, such as conformal predictions. The primary point for this particular thesis project is the reliability of the intervals, at least in the marginal sense. The next is getting small enough intervals to be useful for alerting. Context Account managers in ING Wholesale Banking handle dozens of large corporate clients. For each client they need to be aware of their client’s financial status and movements, so they can for example check if mutual agreements are still upheld, or if there is opportunity for up- or cross-selling. If a client takes on an abnormally large or small account balance position, then this can be an alert for the account manager to take a closer look at the client. To automate these alerts, the WBAA department is modeling abnormal behavior, by first modeling the normal expected range of the client’s balance development and then determining if the actual balance significantly deviates out of this range. Hence the desire for confidence bands on time series predictions of account balances. Research goal We are agnostic to the model or methodology that will be researched, but given the diversity of the account balances and given the past research that we have done, we do believe that classical models, such as ARIMA and linear state space models, are insufficient for this task. We have a number of potential solutions in mind: machine learning + conformal predictions, machine learning + block bootstrap sampling of residuals, training a neural network with a double pinball loss, or simply trying foundation models, but it is up to the student (+ supervisor) to choose and focus on one or more methods. The Foreseen End-result Is a Set Of Measures That State How Well The Chosen Model(s) Can Forecast Reliable Confidence Or Uncertainty Intervals And How Accurate They Are In Comparison With a Set Of Baseline Models (comparing Widths Of Intervals). This Will Include Assessing potential solutions via literature research and/or short experiments and ultimately deciding on one or more solutions to understand, implement and test Implementing the chosen solutions as well as one or more baseline methods (e.g. auto-ARIMA, simple bootstrapping, out-of-the-box deep learner) Choosing metrics that measure 1) the reliability of the produced intervals and 2) the width of the produced intervals Implementing a pipeline to be able to test and compare solutions out-of-time Analyzing for which balance behaviors the solutions do and do not produce reliable intervals (error analysis), and potentially using this to improve the solution Producing a final comparison between solutions and baseline methods on a holdout set The team The Transaction Services (TS) team of WBAA at ING provides data analysis and data science solutions for the TS department. The TS department is interested in using forecasts of account balances to improve their client interactions and decision making. The WBAA department is a large team of data scientists, data engineers, software developers and many more, that are focused on bringing data, machine learning and statistical modeling into the products that we build for our clients or internal users. The data scientists in WBAA furthermore have a strong desire to keep up with and be part of the latest developments in the fields of AI, tooling and statistics. Which they do by working closely together with master’s students on a variety of topics to solve academic yet practical problems. Our team has extensive experience with student supervision. Are you a master’s student looking for a thesis project and are you interested in this one. How To Succeed We hire smart people like you for your potential. Our biggest expectation is that you’ll stay curious. Keep learning. Take on responsibility. In return, we’ll back you to develop into an even more awesome version of yourself. To take on this challenging and rewarding opportunity, you’ll need to: Have solid experience with Python Have machine learning experience Have solid skills in statistics and linear algebra (matrix rank, singular values, matrix decomposition, …) Get at least six months to do your thesis project Aim to go for a publication Bring good vibes to your fellow data scientists What do we offer? A master thesis project, a compensation of 700 euros per month, close supervision, and a tight community of data scientists to interact with and learn from. Rewards And Benefits This is a great opportunity to train with highly skilled people who are experts in their field. You’ll do a lot and learn a lot – not only about your specialist area and the bank, but also about yourself and whether this type of environment is right for you. You’ll Also Benefit From Internship allowance of 700 EUR based on 36 hours work week Your own work laptop Hybrid working to blend home working for focus and office working for collaboration and co-creation Personal growth and challenging work with endless possibilities An informal working environment with innovative colleagues During the duration of your internship at ING, it is mandatory to be enrolled at a Dutch university (or EU-university for EU passport holders). Questions? Contact the recruiter attached to the advertisement. Want to apply directly? Please upload your CV and motivation letter by clicking the ‘Apply’ button. About Our Internships Every year, more than 350 students join our internship program. While there are no guarantees about your future, many of our former interns move into a permanent role or onto our International Talent Programme (traineeship). Whatever happens, an internship at ING is the ideal opportunity to meet a wide variety of people, to build up your own network, and to learn about many different aspects of banking – put simply, it’s a great start to your career.

Core Responsibilities

Research, implement, and compare methods for forecasting reliable confidence or uncertainty intervals for corporate account balances, including selected approaches and baseline models. Build an out-of-time testing pipeline, evaluate interval reliability and width, analyze cases where forecasts are unreliable, and report final results on a holdout set.

Requirements

Applicants should have solid Python experience, machine learning experience, and strong skills in statistics and linear algebra. The project requires at least six months, and the student should be enrolled at a Dutch university or, if holding an EU passport, an EU university during the internship; aiming for a publication is encouraged.

Benefits

• Internship Allowance
• Work Laptop
• Hybrid Working
• Personal Growth Opportunities
• Challenging Work
• Informal Working Environment
• Close Supervision
• Community Of Data Scientists