Long-term use of the hybrid artificial pancreas by adjusting carbohydrate ratios and programmed basal rate: A reinforcement learning approach

• Published in: Computer methods and programs in biomedicine 2021-03, Vol.200, p.105936-105936, Article 105936
• Main Authors: Jafar, Adnan, Fathi, Anas El, Haidar, Ahmad
• Format: Article
• Language: English
• Subjects: Adaptation; Algorithms; Blood Glucose; Carbohydrate ratio; Diabetes Mellitus, Type 1 - drug therapy; Humans; Hybrid artificial pancreas; Hypoglycemic Agents - therapeutic use; Insulin - therapeutic use; Insulin Infusion Systems; Pancreas, Artificial; Reinforcement learning; Type 1 diabetes
• ISSN: 0169-2607; 1872-7565
• DOI: 10.1016/j.cmpb.2021.105936

•Hybrid artificial pancreas uses carbohydrate ratios and programmed basal rate that can vary significantly due to large intra- and inter-patient variabilities that may result in over-or under-delivery of insulin by the hybrid artificial pancreas.•We propose a novel method to adapt carbohydrate ratios and programmed basal rate in the hybrid artificial pancreas to improve blood glucose control.•Use of a reinforcement learning in the hybrid artificial pancreas can improve patients’ quality of life by reducing the risk of severe and non-severe hypoglycemia.•Use of the reinforcement learning method in the hybrid artificial pancreas increases the time spent in target range.•Proposed method is a key in making the hybrid artificial pancreas adaptive for the long-term real-life outpatient studies. The hybrid artificial pancreas regulates glucose levels in people with type 1 diabetes. It delivers (i) insulin boluses at meal times based on the meals’ carbohydrate content and the carbohydrate ratios (CRs) and (ii) insulin basal, between meals and at night, continuously modulated around individual-specific programmed basal rate. The CRs and programmed basal rate significantly vary between individuals and within the same individual with type 1 diabetes, and using suboptimal values in the hybrid artificial pancreas may degrade glucose control. We propose a reinforcement learning algorithm to adaptively optimize CRs and programmed basal rate to improve the performance of the hybrid artificial pancreas. The proposed reinforcement learning algorithm was designed using the Q-learning approach. The algorithm learns the optimal actions (CRs and programmed basal rate) by applying them to the individual's state (previous day's glucose levels and insulin delivery) based on an exploration and exploitation trade-off. First, outcomes from our simulator were compared to those of a clinical study in 23 individuals with type 1 diabetes and have yielded similar results. Second, the learning algorithm was tested using the simulator with two scenarios. Scenario 1 has fixed meal sizes and ingestion times and scenario 2 has a more realistic eating behavior with random meal sizes, ingestion times, and carbohydrate counting errors. After about five weeks, the reinforcement learning algorithm improved the percentage of time spent in target range from 67% to 86.7% in scenario 1 and 65.5% to 86% in scenario 2. The percentage of time spent below 4.0 mmol/L decreased from 9% to 0.9% in scenario 1