High school student uses AI to 'read doctor's handwriting', digitizing thousands of patient records

In just 20 seconds, Gia Minh and his friends' AI application can interpret doctor's handwriting and automatically input data, digitizing approximately 5,000 school dental examination records to facilitate treatment tracking.

Nguyen Hoang Gia Minh is a 12th-grade student at Newton School, Hanoi. He and his friends developed the SmileScan system to automatically input patient information for Operation Smile Vietnam, a non-governmental organization providing free surgeries for children and adolescents with congenital craniofacial deformities such as cleft lip and palate.

The system uses a phone camera to scan and recognize handwriting on patient records, automatically populating data fields for storage. This process takes about 20 seconds per record. Additionally, the system automatically generates charts, showing the number of treated patients and common dental issues.

"Our idea is to help medical professionals reduce their working time and easily monitor treatment progress, without the effort of searching for records," Gia Minh said.

Nguyen Hoang Gia Minh. Photo: Family provided

Minh stated that he has participated in volunteer activities since 9th grade, starting with supporting the setup of mobile clinics at Operation Smile Vietnam. The student mentioned that each examination round lasts four days, serving 2,500 to 4,000 children. For surgical rounds, out of about 270 records, 130-150 children are selected. After observing for some time, Minh realized that medical staff still stored records on paper or typed patient records into Excel spreadsheets.

"This makes it difficult for the organization, medical staff, and volunteers to compile statistics, assess patient history, and generate related medical reports," Minh observed.

Therefore, since September last year, Minh and his team developed an application integrating artificial intelligence, using a large language model to scan, recognize handwriting, and automatically input information into a database.

The biggest challenge, according to Minh, was recognizing "doctor's handwriting" and specialized medical terminology.

"Each doctor has a unique writing style and uses many terms, so without medical knowledge, it's impossible to provide accurate commands," Minh recounted, giving examples like "boi veni" meaning "boi vecni fluor" (applying fluoride varnish) or the letter "F" meaning "boi Fluoride" (applying fluoride).

To overcome this, he and his friend compiled a table of frequently used medical terms and photographed nearly 2,500 patient records for training data.

"The goal is for anyone, even regular logistics volunteers without medical expertise, to be able to use it for data entry," Minh said.

After several refinements and expert support, the system achieved 90% accuracy, verified during data entry from examination rounds in Quang Ngai and Hue in May. The project then won an outstanding presentation award at a young scientific research conference in Japan.

On its website, Operation Smile Vietnam stated that the project has digitized approximately 5,000 handwritten records, commending the students' efforts to apply technology in solving practical public health challenges.

The system will continue to be developed and integrated into Operation Smile Vietnam's digitization projects over the next two years, with a USD 35,000 grant from Amazon.

In the future, Minh and his friends plan to address the challenge of digitizing surgical records.

"These records contain extensive information, going through multiple processes from hospital admission and screening to diagnosis, so consolidating data into a single point is a significant challenge," Minh shared, expressing hope to expand the system to be multilingual for synchronous application in dental examination programs in other countries.