Improvements in technology have allowed researchers to develop artificial pancreas systems (APS) that mimic glucose regulation function of a healthy pancreas. This study aims to develop a closed-loop APS to control blood glucose level (BGL) of diabetic rats in vivo. To this end, we developed a customized APS with a control algorithm based on model predictive control (MPC). MPC is widely used in many studies to determine the optimal insulin infusion based on BGL projection. We developed a time-series artificial neural network (ANN) to make multistep ahead predictions for MPC based on past BGL, insulin injection, and food intake. The objective of the ANN-based MPC (NN-MPC) is to keep BGL of the rats in vivo around a target value and maintain it within a normal range. The NN-MPC was tested in 24-hour experiments with type 1 diabetic rats monitored and controlled by our customized APS. For evaluation, the performance of the NN-MPC was compared with that of the state-of-the-art OpenAPS. Experimental results show that NN-MPC outperformed OpenAPS in terms of time within the normal range (88.47% vs. 71.82%) and variation of BGL. The customized APS with NN-MPC showed promising results for continuous, real-time control of BGL. The system can provide a testbed platform for closed-loop control where various control schemes and parameter tuning can be developed and validated.

Artificial neural network, Artificial pancreas system, Blood glucose level control, Model predictive control, Type 1 diabetes mellitus

Advances in sensor and computing technology have enabled researchers to develop artificial pancreas systems (APS) that can mimic the glucose regulation function of a healthy pancreas to treat type 1 diabetes mellitus (T1DM). APS use a continuous glucose monitor (CGM) sensor to read the latest blood glucose level (BGL). This information is then sent to a control algorithm device (CAD). Information sent to the CAD can be viewed by the patient, but more importantly, the control algorithm computes the required insulin amount to be delivered through an insulin pump. CGM sensor readings are updated every five minutes and new control decisions are implemented by the CAD. Figure 1 illustrates the interactions between the different components in APS. APS have shown to safely improve the life quality of T1DM patients through frequent glucose sampling and insulin adjustments [1]. Many studies have shown that APS have become more popular, and deployment of these devices has increased rapidly [2-6].

Figure 1: Artificial pancreas system.

Control algorithms used in APS play an important role of controlling BGL of T1DM patients. They calculate appropriate insulin doses based on the projected BGL estimated from CGM data. It is challenging due to many factors such as inter-individual variability, time-varying dynamics, nonlinear phenomena, and delays between subcutaneous (SC) level and the blood stream [7,8]. In addition, internal physiological factors and external disturbances such as meals and physical activities affect the glucose-insulin dynamics. Control algorithms presented in the literature can be categorized into the following: physiological model-based control [9-12], fuzzy based control [13-15], proportional integral derivative (PID) control [16-18], and model predictive control (MPC) [19-22]. Among these, MPC is widely accepted thanks to its advantage of handling receding horizon control as a constrained optimization problem; however, MPC suffers from its complexity and requirement of variable initialization, which make its real-time application challenging. This issue has been tackled by replacing the prediction component of MPC with artificial neural networks (ANN) [23]. ANN can generate appropriate models for predicting BGL because of their ability to learn complex nonlinear patterns in glucoseinsulin dynamics based on various inputs. ANN have been successfully applied for BGL prediction and control in literature [23-26]; however, they have been tested in silico with retrospective datasets or in restrictive settings.

As for real-life BGL control, the Open Artificial Pancreas System (Open APS) is considered as one of the most advanced and rapidly evolving APS for closed-loop control [27]. The outcomes of the Open APS control algorithm have been analyzed in recent studies [28,34]. They showed that Open APS could achieve good real-life control performance comparable to that of other APS reported in the literature. Therefore, Open APS has been used in this study as the benchmark to evaluate our customized APS.

The objective of this study is to realize a continuous closedloop system to control BGL of T1DM rats in vivo. To this end, we developed a customized APS on top of Open APS to implement a closed-loop control on the rats. The ANNbased MPC (NN-MPC) is used as the control algorithm which determines the optimal amount of insulin injection based on multi-step ahead predictions made by a time-series ANN. There are other studies that have demonstrated MPC on APS with in vivo subjects in clinical trials [29,30]; however, they have been performed in limited settings and restricted schedules. On the other hand, we tested our APS around the clock with rats which were free to move laterally and consume food ad libitum. The goal of NN-MPC is to maintain BGL of T1DM rats within the normal range (70-180 mg/dl) under this condition. Our APS has been evaluated in comparison with the widely used Open APS based on time within the normal range and variation of BGL.

ANN for BGL prediction

Several studies have shown that ANN can accurately predict BGL in individual subjects considering factors such as insulin injection and food intake [23,26,31]. In our APS, a time series ANN, particularly a nonlinear autoregressive network with exogenous inputs (NARX) ANN, is used for BGL prediction. NARX ANN can learn to predict a time series given past values of the same time series (feedback input) and the external (exogenous) time series. This model is to predict a new output (BGL) based on three sets of variables: past BGL outputs and two external inputs, insulin injection and food intake. Based on these variables, the ANN is set to make BGL predictions every five minutes as shown in the following equation:

where G(t) is BGL at t; X(t) is a 2*d external input matrix that contains insulin injection amount, I(t), and food intake F(t) at t; and d is the number of tapped delays for past inputs used for prediction. Equation (1) represents how the ANN predicts G(t) given d past values of BGL reported by CGM and X(t) reported by insulin pump and food scale. Considering how long food intake and insulin injection might affect BGL (12-15 steps by food and 4-6 steps by insulin in preliminary experiments), d is set to 20. The dataset is collected from each T1DM rat and used to train a NARX ANN with d-steps of previous values taken into consideration. Figure 2 shows the architecture of the ANN with the sources of input data. The feedback loop is open for training so that the ANN can learn from past data whereas it is closed for prediction so that the ANN can make multi-step ahead predictions for MPC. The NARX ANN is used to make predictions for the next 30 minutes (six-time steps), which are adequate for the MPC to determine optimal insulin injection considering the onset time of fast acting insulin.

Figure 2: The NARX ANN architecture.

ANN-based MPC for BGL control

Model predictive control (MPC) is a feedback control algorithm that uses a model to make predictions about future outputs of a process. Once the refence trajectory is defined, MPC optimizes the future inputs and drives the outputs predicted by the model to match the trajectory. In our APS, MPC works on top of the NARX ANN that serves as the prediction component. Based on multi-step ahead predictions of BGL made by the NARX ANN, the NN-MPC can determine the optimal insulin injection by solving the following problem:

where n is the prediction horizon (n = 6 in this study), is the predicted BGL after k steps, is the reference BGL (set to the midpoint of the normal BGL range, ~130 mg/dl), and u_k is the decision variable which is the amount of insulin injection after k steps.  is predicted by the NARX ANN with assumption of no food intake in the near future because it is unknown at the moment of prediction. MPC can smoothen the change in the decision variable by applying an appropriate penalty parameter (ρ) for control move suppression; however, this parameter is set to zero in this study so that the MPC can allow quick bolus injections of insulin. As a safety measure, the maximum insulin injection at each step (k) in the prediction horizon is limited up to 3.2 U/hr. After solving Eq. (2), the NN-MPC implements the first control action (u₁), and the optimization problem is continuously updated and solved to determine the optimal action at every time step (5 minutes). Figure 3 depicts the overall NN-MPC model.

Figure 3: The NN-MPC model.

Configuration of artificial pancreas system

A customized APS was developed on top of Open APS to perform experiments on T1DM rats (Sprague Dawley rats with streptozotocin-induced diabetes; all animal maintenance and treatment protocols complied with the Guide for Care and Use of Laboratory Animals as adopted by the National Institute of Health and approved by the Institutional Animal Care and Use Committee (IACUC) of Southern Illinois University Edwardsville). Figure 4 shows an overview of the APS, which consists of CGM, insulin pump, CAD (Intel Edison computeron- module loaded with Open APS), food scale, and a control server where the NN-MPC is implemented to control BGL of T1DM rats in a restrainer. A CGM sensor was attached behind the front left leg and an insulin needle was inserted subcutaneously to the back near the front right leg. The rat was placed in a specially designed restrainer to limit the range of motion so that the CGM sensor and insulin needle could be kept in place. In order to prevent over stress, rats were kept in the restrainer for 48 hours or less. Three main variables are obtained from the APS for creation of the NN-MPC: BGL, insulin injection amount, and food intake. In each experiment, a dataset of these three variables was collected for a 24-hour period after the CGM was calibrated.

Figure 4: An overview of the customized APS.

Food intake: All rats were allowed to move laterally and eat food (94.8% carbohydrates sugar pellets) and water ad libitum during the 24-hour experiments. We chose the high carbohydrate diet to accentuate the blood and insulin dynamics. To measure food intake automatically, a scale was designed by using a single-board microcontroller and connected to the control server via a serial port to record food consumption every five minutes. A small food storage bin was placed on top of a 5-kg digital load cell weight sensor attached to a HX711 load cell amplifier to measure how much food was taken from the bin. However, the readings can be very noisy since the rats have direct access to the food bin. To make the readings less prone to noise, a 300-mm linear actuator was attached and programmed to pull the load cell with the food bin on top away from the restrainer once every 5 minutes. While the food bin is away, the load cell takes ten measurements, and the average value is recorded as food intake with a timestamp in a text file stored in the control server. After the value is recorded, the food bin returns to its initial position where the rat can have access again. This cycle is repeated every five minutes.

Blood glucose level: BGL values are measured every five minutes by using a CGM (Medtronic Enlite sensor and transmitter). The readings once available are sent to the Intel Edison module that can handle communication between CGM, insulin pump, and the control server. Values of BGL along with insulin infusion are collected and displayed by the Nightscout cloud application (http://www.nightscout.info) with a MongoDB database (https://www.mongodb.com). In addition, the latest 24-hour BGL readings are made available in a JSON (JavaScript Object Notation) formatted file that is delivered to the control server via the Intel Edison module. Figure 5 shows a screenshot of the Nightscout application, where insulin infusion is shown in blue at the top and BGL readings in the middle in green (optimal), yellow (borderline), or red (low).

Figure 5: A screenshot of N ight scout application sho wing the trajectory of BGL and insulin infusion.

Insulin injection: To evaluate the proposed NN-MPC, we performed experiments using the Open APS built-in control algorithm for comparison. During the 24-hour experiments, the amount of insulin injection recommended by Open APS is sent to the insulin pump and recorded in a text file with a timestamp in the Intel Edison module. On the other hand, the NN-MPC resides in the control server and calculates the optimal insulin injection based on the time-series data received through the Intel Edison module. The calculated control input is sent to the insulin pump (Medtronic Minimed Pump 722) via the Intel Edison module every five minutes to inject insulin (Humalog 100 units/ml stock solution diluted to 1:7) to the rat.

A NARX ANN model was created with a hidden layer of 15 neurons and trained by Bayesian regularization backpropagation. Datasets for training the model were collected in preliminary experiments performed with Open APS. The prediction error of NARX ANN was 10-11 mg/ dl for 30-min prediction [23]. It was much lower for 5-min prediction (3-4 mg/dl) and therefore the NARX ANN provided BGL predictions that are accurate enough for the MPC. To evaluate the performance of NN-MPC, a total of eight samples of 24-hour data were collected from eight experiments: four controlled by Open APS and four by NN-MPC. Table 1 summarizes the experimental results, including % time in normal, hypoglycemic (<70 mg/dl) and hyperglycemic (>180 mg/dl) range, mean/max/min BGL, and RMSE between the actual and the target BGL. The results show that the NNMPC performs better than Open APS in terms of all measures except time in hypoglycemia. Particularly, NN-MPC yields a significantly larger time within the normal range than Open APS (88.47% vs. 71.82% with p-value<0.001 by t-test) due to the dramatically reduced time in hyperglycemia (7.39% vs. 27.46%). In addition, NN-MPC controls BGL in a range tighter than that obtained by Open APS as shown by its smaller BGL range and RMSE. As it can be seen from Figure 6 and Table 1, NN-MPC tends to maintain BGL in a lower range around the target BGL; however, this might lead to increased time in hypoglycemia (4.14% vs. 0.71%).

^a G ? = 125 mg/dl;  = 135 mg/dl for the other observations with NN-MPC

Table 1: Summary of experimental results.

Figure 6: Boxplots of BGL controlled by NN-MPC (with ????????? = 135 mg/dl) and Open APS.

A sample comparison of the two control methods is illustrated in Figure 7. Since the rats were allowed to take food ad libitum, the food intake data were very different case by case; however, both methods worked well in controlling BGL as shown in green lines. Figure 7(a) shows the trajectory of BGL and the amount of insulin injected by Open APS along with food intake. The BGL is appropriately controlled by Open APS with minimal hypoglycemia; however, it shows periods of hyperglycemia (between 6 and 11hrs) although the severity is moderate. The insulin injection by Open APS shows gradual changes upward or downward depending on the trend in BGL. On the other hand, NN-MPC frequently gives the maximum insulin injections (3.2 U/hrs.) for short durations as shown in Figure 7(b). This aggressive action is caused by ignoring control move suppression in the objective function of MPC (ρ = 0), which allows prompt bolus injection for immediate control. NN-MPC maintains BGL in the normal range most of the time with some brief periods of hypoglycemia and hyperglycemia. The aggressive control by NN-MPC resulted in short periods of hypoglycemia; however, they are very brief as seen in Figure. 7(b) and can be overcome by tuning some parameters in the model. For example, % time in hypoglycemia has been considerably reduced by simply modifying the target BGL from 125 mg/dl (Obs. 1) to 135 mg/dl (Obs. 2-4) in NN-MPC as shown in Table 1.

Figure 7: A sample result by (a) Open APS and (b) NN-MPC.

The experimental results show that the NN-MPC is able to determine an appropriate amount of insulin injection quickly in response to the trajectory of BGL and keep BGL within the normal range most of the time; however, the aggressive control actions cause BGL to be lowered on average and sometimes lead to hypoglycemia. One solution to this problem was to raise the target BGL as explained in the previous section. After observing this quick modification decreasing % time in hypoglycemia from 9.31% to ~1%, we continued our experiments with  ????????? set to 135 mg/dl, which resulted in BGL shifted upward and led to reduced hypoglycemia (2.41% on average). This demonstrates that simple parameter tuning can significantly improve the performance of the NN-MPC.

There are other potential solutions to improve performance and robustness of NN-MPC. We adopted a simple objective function in MPC without imposing penalty for control move suppression to allow quick bolus injection. However, the penalty parameter could be tuned to a positive value either empirically or by using a regularization technique so that the control would be less abrupt and more conservative. In addition, the objective function can be further modified to enable zone-MPC with asymmetric penalty scheme [32] where the cost function is of an asymmetric form instead of the symmetric form of the target-based control. Since hypoglycemia poses a greater risk than hyperglycemia, the objective function can be modified to impose larger penalties in hypoglycemic zones while tolerating higher BGL.

A high accuracy of the ANN prediction model is the prerequisite for robust control. Inputs to the ANN include food intake as well as BGL and insulin injection; however, the food scale was very sensitive to disturbances (e.g., vibration of the actuator and the restrainer) and gave erroneous values from time to time during the experiments. Even though the NN-MPC achieved a good control performance under the circumstances, having more reliable data for food intake may improve the performance further. To this end, the quality of the food intake data could be augmented by other supplementary techniques such as convolutional neural networks that estimate the amount of food in tray from images. On the other hand, one can consider completely dropping food intake data from the current hybrid closed-loop control and developing the NN-MPC without the feed-forward information in order to fully close the loop and ease the management of T1DM. The feasibility of this unannounced meal condition has been studied in literature. For example, Martinsson et al. [33] reported a deep neural network BGL prediction model that achieved a high accuracy without meal intake data. In addition, other factors that are relatively easy to access can be incorporated in the inputs of ANN, including exercise, stress level, heart rate, body temperature, and circadian rhythm. Feature engineering, e.g., the rate of change in BGL, may help to improve the prediction accuracy. No matter what factors and features are included, the ANN needs to be continuously updated with new datasets that cover a wide range of input variations, particularly in the hypoglycemic zone, in order to reflect intra-subject variability in the metabolism of the subject.

We presented a customized APS with NN-MPC for closedloop BGL control of T1DM rats in vivo in real time. Our APS provided an environment to test control algorithms in a less restrictive and closer to real life setting. The NN-MPC deployed in our APS achieved significantly better performance than the popular Open APS in terms of time within the normal range and variation from the reference level. Quality of input data and other inputs such as exercise level would help improve the prediction accuracy and subsequently the control performance.

The customized APS with NN-MPC showed promising results for continuous, real-time control of BGL in diabetic subjects. Furthermore, this system can provide a testbed platform where various control schemes and parameter tuning can be developed and validated. The successful demonstration of the system is a step-forward to realize a fully automated closedloop control system. In the future, various setups in MPC, including asymmetric penalty scheme and parameter tuning, need to be studied. The impact of additional features that are easily accessible (e.g., circadian rhythm) can be investigated as well. Different BGL prediction methods such as recurrent neural networks with/without meal announcement can be another potential future work.

The authors declare that they have no conflict of interest.