Improvements in technology have allowed researchers to develop artificial pancreas systems (APS) that mimic glucose regulation function of a healthy pancreas. This study aims to develop a closed-loop APS to control blood glucose level (BGL) of diabetic rats in vivo. To this end, we developed a customized APS with a control algorithm based on model predictive control (MPC). MPC is widely used in many studies to determine the optimal insulin infusion based on BGL projection. We developed a time-series artificial neural network (ANN) to make multistep ahead predictions for MPC based on past BGL, insulin injection, and food intake. The objective of the ANN-based MPC (NN-MPC) is to keep BGL of the rats in vivo around a target value and maintain it within a normal range. The NN-MPC was tested in 24-hour experiments with type 1 diabetic rats monitored and controlled by our customized APS. For evaluation, the performance of the NN-MPC was compared with that of the state-of-the-art OpenAPS. Experimental results show that NN-MPC outperformed OpenAPS in terms of time within the normal range (88.47% vs. 71.82%) and variation of BGL. The customized APS with NN-MPC showed promising results for continuous, real-time control of BGL. The system can provide a testbed platform for closed-loop control where various control schemes and parameter tuning can be developed and validated.
Artificial neural network, Artificial pancreas system, Blood glucose level control, Model predictive control, Type 1 diabetes mellitus
Advances in sensor and computing technology have enabled researchers to develop artificial pancreas systems (APS) that can mimic the glucose regulation function of a healthy pancreas to treat type 1 diabetes mellitus (T1DM). APS use a continuous glucose monitor (CGM) sensor to read the latest blood glucose level (BGL). This information is then sent to a control algorithm device (CAD). Information sent to the CAD can be viewed by the patient, but more importantly, the control algorithm computes the required insulin amount to be delivered through an insulin pump. CGM sensor readings are updated every five minutes and new control decisions are implemented by the CAD. Figure 1 illustrates the interactions between the different components in APS. APS have shown to safely improve the life quality of T1DM patients through frequent glucose sampling and insulin adjustments . Many studies have shown that APS have become more popular, and deployment of these devices has increased rapidly [2-6].
Figure 1: Artificial pancreas system.
Control algorithms used in APS play an important role of controlling BGL of T1DM patients. They calculate appropriate insulin doses based on the projected BGL estimated from CGM data. It is challenging due to many factors such as inter-individual variability, time-varying dynamics, nonlinear phenomena, and delays between subcutaneous (SC) level and the blood stream [7,8]. In addition, internal physiological factors and external disturbances such as meals and physical activities affect the glucose-insulin dynamics. Control algorithms presented in the literature can be categorized into the following: physiological model-based control [9-12], fuzzy based control [13-15], proportional integral derivative (PID) control [16-18], and model predictive control (MPC) [19-22]. Among these, MPC is widely accepted thanks to its advantage of handling receding horizon control as a constrained optimization problem; however, MPC suffers from its complexity and requirement of variable initialization, which make its real-time application challenging. This issue has been tackled by replacing the prediction component of MPC with artificial neural networks (ANN) . ANN can generate appropriate models for predicting BGL because of their ability to learn complex nonlinear patterns in glucoseinsulin dynamics based on various inputs. ANN have been successfully applied for BGL prediction and control in literature [23-26]; however, they have been tested in silico with retrospective datasets or in restrictive settings.
As for real-life BGL control, the Open Artificial Pancreas System (Open APS) is considered as one of the most advanced and rapidly evolving APS for closed-loop control . The outcomes of the Open APS control algorithm have been analyzed in recent studies [28,34]. They showed that Open APS could achieve good real-life control performance comparable to that of other APS reported in the literature. Therefore, Open APS has been used in this study as the benchmark to evaluate our customized APS.
The objective of this study is to realize a continuous closedloop system to control BGL of T1DM rats in vivo. To this end, we developed a customized APS on top of Open APS to implement a closed-loop control on the rats. The ANNbased MPC (NN-MPC) is used as the control algorithm which determines the optimal amount of insulin injection based on multi-step ahead predictions made by a time-series ANN. There are other studies that have demonstrated MPC on APS with in vivo subjects in clinical trials [29,30]; however, they have been performed in limited settings and restricted schedules. On the other hand, we tested our APS around the clock with rats which were free to move laterally and consume food ad libitum. The goal of NN-MPC is to maintain BGL of T1DM rats within the normal range (70-180 mg/dl) under this condition. Our APS has been evaluated in comparison with the widely used Open APS based on time within the normal range and variation of BGL.
Materials and Methods
ANN for BGL prediction
Several studies have shown that ANN can accurately predict BGL in individual subjects considering factors such as insulin injection and food intake [23,26,31]. In our APS, a time series ANN, particularly a nonlinear autoregressive network with exogenous inputs (NARX) ANN, is used for BGL prediction. NARX ANN can learn to predict a time series given past values of the same time series (feedback input) and the external (exogenous) time series. This model is to predict a new output (BGL) based on three sets of variables: past BGL outputs and two external inputs, insulin injection and food intake. Based on these variables, the ANN is set to make BGL predictions every five minutes as shown in the following equation:
where G(t) is BGL at t; X(t) is a 2*d external input matrix that contains insulin injection amount, I(t), and food intake F(t) at t; and d is the number of tapped delays for past inputs used for prediction. Equation (1) represents how the ANN predicts G(t) given d past values of BGL reported by CGM and X(t) reported by insulin pump and food scale. Considering how long food intake and insulin injection might affect BGL (12-15 steps by food and 4-6 steps by insulin in preliminary experiments), d is set to 20. The dataset is collected from each T1DM rat and used to train a NARX ANN with d-steps of previous values taken into consideration. Figure 2 shows the architecture of the ANN with the sources of input data. The feedback loop is open for training so that the ANN can learn from past data whereas it is closed for prediction so that the ANN can make multi-step ahead predictions for MPC. The NARX ANN is used to make predictions for the next 30 minutes (six-time steps), which are adequate for the MPC to determine optimal insulin injection considering the onset time of fast acting insulin.
Figure 2: The NARX ANN architecture.
ANN-based MPC for BGL control
Model predictive control (MPC) is a feedback control algorithm that uses a model to make predictions about future outputs of a process. Once the refence trajectory is defined, MPC optimizes the future inputs and drives the outputs predicted by the model to match the trajectory. In our APS, MPC works on top of the NARX ANN that serves as the prediction component. Based on multi-step ahead predictions of BGL made by the NARX ANN, the NN-MPC can determine the optimal insulin injection by solving the following problem:
where n is the prediction horizon (n = 6 in this study), is the predicted BGL after k steps, is the reference BGL (set to the midpoint of the normal BGL range, ~130 mg/dl), and uk is the decision variable which is the amount of insulin injection after k steps. is predicted by the NARX ANN with assumption of no food intake in the near future because it is unknown at the moment of prediction. MPC can smoothen the change in the decision variable by applying an appropriate penalty parameter (ρ) for control move suppression; however, this parameter is set to zero in this study so that the MPC can allow quick bolus injections of insulin. As a safety measure, the maximum insulin injection at each step (k) in the prediction horizon is limited up to 3.2 U/hr. After solving Eq. (2), the NN-MPC implements the first control action (u1), and the optimization problem is continuously updated and solved to determine the optimal action at every time step (5 minutes). Figure 3 depicts the overall NN-MPC model.
Figure 3: The NN-MPC model.
Configuration of artificial pancreas system
A customized APS was developed on top of Open APS to perform experiments on T1DM rats (Sprague Dawley rats with streptozotocin-induced diabetes; all animal maintenance and treatment protocols complied with the Guide for Care and Use of Laboratory Animals as adopted by the National Institute of Health and approved by the Institutional Animal Care and Use Committee (IACUC) of Southern Illinois University Edwardsville). Figure 4 shows an overview of the APS, which consists of CGM, insulin pump, CAD (Intel Edison computeron- module loaded with Open APS), food scale, and a control server where the NN-MPC is implemented to control BGL of T1DM rats in a restrainer. A CGM sensor was attached behind the front left leg and an insulin needle was inserted subcutaneously to the back near the front right leg. The rat was placed in a specially designed restrainer to limit the range of motion so that the CGM sensor and insulin needle could be kept in place. In order to prevent over stress, rats were kept in the restrainer for 48 hours or less. Three main variables are obtained from the APS for creation of the NN-MPC: BGL, insulin injection amount, and food intake. In each experiment, a dataset of these three variables was collected for a 24-hour period after the CGM was calibrated.
Figure 4: An overview of the customized APS.
Food intake: All rats were allowed to move laterally and eat food (94.8% carbohydrates sugar pellets) and water ad libitum during the 24-hour experiments. We chose the high carbohydrate diet to accentuate the blood and insulin dynamics. To measure food intake automatically, a scale was designed by using a single-board microcontroller and connected to the control server via a serial port to record food consumption every five minutes. A small food storage bin was placed on top of a 5-kg digital load cell weight sensor attached to a HX711 load cell amplifier to measure how much food was taken from the bin. However, the readings can be very noisy since the rats have direct access to the food bin. To make the readings less prone to noise, a 300-mm linear actuator was attached and programmed to pull the load cell with the food bin on top away from the restrainer once every 5 minutes. While the food bin is away, the load cell takes ten measurements, and the average value is recorded as food intake with a timestamp in a text file stored in the control server. After the value is recorded, the food bin returns to its initial position where the rat can have access again. This cycle is repeated every five minutes.
Figure 5: A screenshot of N ight scout application sho wing the trajectory of BGL and insulin infusion.
Insulin injection: To evaluate the proposed NN-MPC, we performed experiments using the Open APS built-in control algorithm for comparison. During the 24-hour experiments, the amount of insulin injection recommended by Open APS is sent to the insulin pump and recorded in a text file with a timestamp in the Intel Edison module. On the other hand, the NN-MPC resides in the control server and calculates the optimal insulin injection based on the time-series data received through the Intel Edison module. The calculated control input is sent to the insulin pump (Medtronic Minimed Pump 722) via the Intel Edison module every five minutes to inject insulin (Humalog 100 units/ml stock solution diluted to 1:7) to the rat.
A NARX ANN model was created with a hidden layer of 15 neurons and trained by Bayesian regularization backpropagation. Datasets for training the model were collected in preliminary experiments performed with Open APS. The prediction error of NARX ANN was 10-11 mg/ dl for 30-min prediction . It was much lower for 5-min prediction (3-4 mg/dl) and therefore the NARX ANN provided BGL predictions that are accurate enough for the MPC. To evaluate the performance of NN-MPC, a total of eight samples of 24-hour data were collected from eight experiments: four controlled by Open APS and four by NN-MPC. Table 1 summarizes the experimental results, including % time in normal, hypoglycemic (<70 mg/dl) and hyperglycemic (>180 mg/dl) range, mean/max/min BGL, and RMSE between the actual and the target BGL. The results show that the NNMPC performs better than Open APS in terms of all measures except time in hypoglycemia. Particularly, NN-MPC yields a significantly larger time within the normal range than Open APS (88.47% vs. 71.82% with p-value<0.001 by t-test) due to the dramatically reduced time in hyperglycemia (7.39% vs. 27.46%). In addition, NN-MPC controls BGL in a range tighter than that obtained by Open APS as shown by its smaller BGL range and RMSE. As it can be seen from Figure 6 and Table 1, NN-MPC tends to maintain BGL in a lower range around the target BGL; however, this might lead to increased time in hypoglycemia (4.14% vs. 0.71%).
|Control||Obs||Normal (%)||Hypoglycemia (%)||Hyperglycemia (%)||Mean (mg/dl)||Max (mg/dl)||Min (mg/dl)||RMSE (mg/dl)|
|2 G ?||88.19||1.04||10.76||130||312||68||47.31|
a G ? = 125 mg/dl; = 135 mg/dl for the other observations with NN-MPC
Table 1: Summary of experimental results.
Figure 6: Boxplots of BGL controlled by NN-MPC (with ????????? = 135 mg/dl) and Open APS.
A sample comparison of the two control methods is illustrated in Figure 7. Since the rats were allowed to take food ad libitum, the food intake data were very different case by case; however, both methods worked well in controlling BGL as shown in green lines. Figure 7(a) shows the trajectory of BGL and the amount of insulin injected by Open APS along with food intake. The BGL is appropriately controlled by Open APS with minimal hypoglycemia; however, it shows periods of hyperglycemia (between 6 and 11hrs) although the severity is moderate. The insulin injection by Open APS shows gradual changes upward or downward depending on the trend in BGL. On the other hand, NN-MPC frequently gives the maximum insulin injections (3.2 U/hrs.) for short durations as shown in Figure 7(b). This aggressive action is caused by ignoring control move suppression in the objective function of MPC (ρ = 0), which allows prompt bolus injection for immediate control. NN-MPC maintains BGL in the normal range most of the time with some brief periods of hypoglycemia and hyperglycemia. The aggressive control by NN-MPC resulted in short periods of hypoglycemia; however, they are very brief as seen in Figure. 7(b) and can be overcome by tuning some parameters in the model. For example, % time in hypoglycemia has been considerably reduced by simply modifying the target BGL from 125 mg/dl (Obs. 1) to 135 mg/dl (Obs. 2-4) in NN-MPC as shown in Table 1.
Figure 7: A sample result by (a) Open APS and (b) NN-MPC.
The experimental results show that the NN-MPC is able to determine an appropriate amount of insulin injection quickly in response to the trajectory of BGL and keep BGL within the normal range most of the time; however, the aggressive control actions cause BGL to be lowered on average and sometimes lead to hypoglycemia. One solution to this problem was to raise the target BGL as explained in the previous section. After observing this quick modification decreasing % time in hypoglycemia from 9.31% to ~1%, we continued our experiments with ????????? set to 135 mg/dl, which resulted in BGL shifted upward and led to reduced hypoglycemia (2.41% on average). This demonstrates that simple parameter tuning can significantly improve the performance of the NN-MPC.
There are other potential solutions to improve performance and robustness of NN-MPC. We adopted a simple objective function in MPC without imposing penalty for control move suppression to allow quick bolus injection. However, the penalty parameter could be tuned to a positive value either empirically or by using a regularization technique so that the control would be less abrupt and more conservative. In addition, the objective function can be further modified to enable zone-MPC with asymmetric penalty scheme  where the cost function is of an asymmetric form instead of the symmetric form of the target-based control. Since hypoglycemia poses a greater risk than hyperglycemia, the objective function can be modified to impose larger penalties in hypoglycemic zones while tolerating higher BGL.
A high accuracy of the ANN prediction model is the prerequisite for robust control. Inputs to the ANN include food intake as well as BGL and insulin injection; however, the food scale was very sensitive to disturbances (e.g., vibration of the actuator and the restrainer) and gave erroneous values from time to time during the experiments. Even though the NN-MPC achieved a good control performance under the circumstances, having more reliable data for food intake may improve the performance further. To this end, the quality of the food intake data could be augmented by other supplementary techniques such as convolutional neural networks that estimate the amount of food in tray from images. On the other hand, one can consider completely dropping food intake data from the current hybrid closed-loop control and developing the NN-MPC without the feed-forward information in order to fully close the loop and ease the management of T1DM. The feasibility of this unannounced meal condition has been studied in literature. For example, Martinsson et al.  reported a deep neural network BGL prediction model that achieved a high accuracy without meal intake data. In addition, other factors that are relatively easy to access can be incorporated in the inputs of ANN, including exercise, stress level, heart rate, body temperature, and circadian rhythm. Feature engineering, e.g., the rate of change in BGL, may help to improve the prediction accuracy. No matter what factors and features are included, the ANN needs to be continuously updated with new datasets that cover a wide range of input variations, particularly in the hypoglycemic zone, in order to reflect intra-subject variability in the metabolism of the subject.
We presented a customized APS with NN-MPC for closedloop BGL control of T1DM rats in vivo in real time. Our APS provided an environment to test control algorithms in a less restrictive and closer to real life setting. The NN-MPC deployed in our APS achieved significantly better performance than the popular Open APS in terms of time within the normal range and variation from the reference level. Quality of input data and other inputs such as exercise level would help improve the prediction accuracy and subsequently the control performance.
The customized APS with NN-MPC showed promising results for continuous, real-time control of BGL in diabetic subjects. Furthermore, this system can provide a testbed platform where various control schemes and parameter tuning can be developed and validated. The successful demonstration of the system is a step-forward to realize a fully automated closedloop control system. In the future, various setups in MPC, including asymmetric penalty scheme and parameter tuning, need to be studied. The impact of additional features that are easily accessible (e.g., circadian rhythm) can be investigated as well. Different BGL prediction methods such as recurrent neural networks with/without meal announcement can be another potential future work.
Conflict of Interests
The authors declare that they have no conflict of interest.
2. Allen N, Gupta A. Current diabetes technology: striving for the artificial pancreas. Diagnostics. 2019 Mar;9(1):31.
3. Deshpande S, Pinsker JE, Zavitsanou S, Shi D, Tompot R, Church MM, et al. Design and clinical evaluation of the Interoperable Artificial Pancreas System (iAPS) smartphone app: interoperable components with modular design for progressive artificial pancreas research and development. Diabetes Technology and Therapeutics. 2019 Jan 1;21(1):35-43.
4. Kovatchev B. A century of diabetes technology: signals, models, and artificial pancreas control. Trends in Endocrinology and Metabolism. 2019 Jul 1;30(7):432-44.
5. Lewis DM, Swain RS, Donner TW. Improvements in A1C and timein- range in DIY closed-loop (Open APS) users.
6. Ramli R, Reddy M, Oliver N. Artificial pancreas: current progress and future outlook in the treatment of type 1 diabetes. Drugs. 2019 Jul;79(10):1089-101.
7. Abadi SE, Balouchzadeh R, Uzun G, Ko HS, Lee HF, Park S, et al. Tracking changes of the parameters of glucose-insulin homeostasis during the course of obesity in B6D2F1 mice. Heliyon. 2020 Jan 1;6(1): e03251.
8. Brenner M, Abadi SE, Balouchzadeh R, Lee HF, Ko HS, Johns M, et al. Estimation of insulin secretion, glucose uptake by tissues, and liver handling of glucose using a mathematical model of glucoseinsulin homeostasis in lean and obese mice. Heliyon. 2017 Jun 1;3(6): e00310.
9. Chen CL, Tsai HW. Modeling the physiological glucose–insulin system on normal and diabetic subjects. Computer Methods and Programs in Biomedicine. 2010 Feb 1;97(2):130-40.
10. Dalla Man C, Rizza RA, Cobelli C. Meal simulation model of the glucose-insulin system. IEEE Transactions on Biomedical Engineering. 2007 Sep 17;54(10):1740-9.
11. Finan DA, Palerm CC, Doyle III FJ, Seborg DE, Zisser H, Bevier WC, et al. Effect of input excitation on the quality of empirical dynamic models for type 1 diabetes. AIChE Journal. 2009 May;55(5):1135-46.
12. Hovorka R, Canonico V, Chassin LJ, Haueter U, Massi-Benedetti M, Federici MO, et al. Nonlinear model predictive control of glucose concentration in subjects with type 1 diabetes. Physiological Measurement. 2004 Jul 22;25(4):905.
13. Campos-Delgado DU, Hernández-Ordoñez M, Femat R, Gordillo- Moscoso A. Fuzzy-based controller for glucose regulation in type- 1 diabetic patients by subcutaneous route. IEEE Transactions on Biomedical Engineering. 2006 Oct 16;53(11):2201-10.
14. Ibbini MS, Masadeh MA. A fuzzy logic based closed-loop control system for blood glucose level regulation in diabetics. Journal of Medical Engineering and Technology. 2005 Jan 1;29(2):64-9.
15. Mauseth R, Hirsch IB, Bollyky J, Kircher R, Matheson D, Sanda S, et al. Use of a “fuzzy logic” controller in a closed-loop artificial pancreas. Diabetes Technology and Therapeutics.2013 Aug 1;15(8):628-33.
16. Chee F, Fernando TL, Savkin AV, Van Heeden V. Expert PID control system for blood glucose control in critically ill patients. IEEE Transactions on Information Technology in Biomedicine. 2003 Dec;7(4):419-25.
17. Marchetti G, Barolo M, Jovanovic L, Zisser H, Seborg DE. An improved PID switching control strategy for type 1 diabetes. IEEE Transactions on Biomedical Engineering. 2008 Feb 15;55(3):857-65.
18. Steil GM. Algorithms for a closed-loop artificial pancreas: the case for proportional-integral-derivative control. Journal of Diabetes Science and Technology. 2013 Nov;7(6):1621-31.
19. Bequette BW. Algorithms for a closed-loop artificial pancreas: the case for model predictive control. Journal of Diabetes Science and Technology. 2013 Nov;7(6):1632-43.
20. Lee H, Bequette BW. A closed-loop artificial pancreas based on model predictive control: Human-friendly identification and automatic meal disturbance rejection. Biomedical Signal Processing and Control. 2009 Oct 1;4(4):347-54.
21. Patek SD, Bequette BW, Breton M, Buckingham BA, Dassau E, Doyle III FJ, et al. In silico preclinical trials: methodology and engineering guide to closed-loop control in type 1 diabetes mellitus. Journal of Diabetes Science and Technology. 2009 Mar;3(2):269-82.
22. Schlotthauer G, Gamero LG, Torres ME, Nicolini GA. Modeling, identification and nonlinear model predictive control of type I diabetic patient. Medical Engineering and Physics. 2006 Apr 1;28(3):240-50.
23. Bahremand S, Ko HS, Balouchzadeh R, Lee HF, Park S, Kwon G. Neural network-based model predictive control for type 1 diabetic rats on artificial pancreas system. Medical and Biological Engineering and Computing. 2019 Jan 1;57(1):177-91.
24. de Canete JF, Gonzalez-Perez S, Ramos-Diaz JC. Artificial neural networks for closed loop control of in silico and ad hoc type 1 diabetes. Computer Methods and Programs in Biomedicine.2012 Apr 1;106(1):55-66.
25. Li K, Liu C, Zhu T, Herrero P, Georgiou P. GluNet: A deep learning framework for accurate glucose forecasting. IEEE Journal of Biomedical and Health Informatics. 2019 Jul 29;24(2):414-23.
26. Pérez-Gandía C, Facchinetti A, Sparacino G, Cobelli C, Gómez EJ, Rigla M, et al. Artificial neural network algorithm for online glucose prediction from continuous glucose monitoring. Diabetes Technology and Therapeutics. 2010 Jan 1;12(1):81-8.
27. Lewis DM. Do-it-yourself artificial pancreas system and the Open APS movement. Endocrinology and Metabolism Clinics. 2020 Mar 1;49(1):203-13.
28. Melmer A, Züger T, Lewis DM, Leibrand S, Stettler C, Laimer M. Glycaemic control in individuals with type 1 diabetes using an opensource artificial pancreas system (Open APS). Diabetes, Obesity and Metabolism. 2019 Oct;21(10):2333-7.
29. Song L, Liu C, Yang W, Zhang J, Kong X, Zhang B, et al. Glucose outcomes of a learning-type artificial pancreas with an unannounced meal in type 1 diabetes. Computer Methods and Programs in Biomedicine. 2020 Jul 1;191: 105416.
30. Wang Y, Zhang J, Zeng F, Wang N, Chen X, Zhang B, et al. “Learning” can improve the blood glucose control performance for type 1 diabetes mellitus. Diabetes Technology and Therapeutics. 2017 Jan 1;19(1):41-8.
31. Pappada SM, Cameron BD, Rosman PM, Bourey RE, Papadimos TJ, Olorunto W, et al. Neural network-based real-time prediction of glucose in patients with insulin-dependent diabetes. Diabetes Technology and Therapeutics. 2011 Feb 1;13(2):135-41.
32. Liu SW, Huang HP, Lin CH, Chien IL. A hybrid neural network model predictive control with zone penalty weights for type 1 diabetes mellitus. Industrial and Engineering Chemistry Research. 2012 Jul 4;51(26):9041-60.
33. Martinsson J, Schliep A, Eliasson B, Mogren O. Blood glucose prediction with variance estimation using recurrent neural networks. Journal of Healthcare Informatics Research. 2020 Mar;4(1):1-8.
34. Asarani NA, Reynolds AN, Elbalshy M, Burnside M, de Bock M, Lewis DM, Wheeler BJ. Efficacy, safety, and user experience of DIY or open-source artificial pancreas systems: a systematic review. Acta Diabetologica. 2021;58(5):539–47.