Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Data 2.1 Data Sources 2.1 Data Sources 2.2 SS and SI Categories 2.2 SS and SI Categories 3 Methods 3.1 Lexicon Creation and Expansion 3.1 Lexicon Creation and Expansion 3.2 Annotations 3.2 Annotations 3.3 System Description 3.3 System Description 4 Results 4.1 Demographics and 4.2 System Performance 4.1 Demographics and 4.2 System Performance 5 Discussion 5 Discussion 5.1 Limitations 5.1 Limitations 6 Conclusion, Reproducibility, Funding, Acknowledgments, Author Contributions, and References 6 Conclusion, Reproducibility, Funding, Acknowledgments, Author Contributions, and References SUPPLEMENTARY SUPPLEMENTARY Guidelines for Annotating Social Support and Social Isolation in Clinical Notes Guidelines for Annotating Social Support and Social Isolation in Clinical Notes Other Supervised Models Other Supervised Models References References GUIDELINES FOR ANNOTATING SOCIAL SUPPORT AND SOCIAL ISOLATION IN CLINICAL NOTES Social isolation (SI) is a state of little or no contact or support between an individual and their community. Social support (SS) describes the physical and emotional comfort derived from one’s family, friends, colleagues, and community. Although electronic health records (EHRs) capture SI and SS within the narrative text, manual abstraction of such data is time-consuming and labor-intensive. Research objective To develop a hybrid approach using rule-based and machine learning (ML)-based natural language processing (NLP) for detecting patient-reported SS and SI in EHRs. Annotation task overview The purpose of annotating clinical notes is to highlight key terms and phrases that characterize one’s social standing and, based on their context, make a clinical determination of an individual’s present or past status with respect to SI and SS. Annotation process Guideline development phase: Initially, two annotators will annotate fifty (50) notes. Then, inter-annotator agreement (IAA) will be calculated, and the annotation guidelines will be revised. This process will continue until a sufficiently high agreement (IAA ≥ 0.6) is achieved. The mismatched cases will be adjudicated by iterative discussion until a consensus is reached. The first 50 clinical notes will be added to the gold standard but excluded from the final IAA computation. Guideline development phase: Annotation Sites: Weill Cornell Medicine (WCM) and Mount Sinai Health System (MSHS). Annotation Sites: Annotation Phase: Each document will be independently annotated by two annotators. After the annotations are completed, the conflicts will be resolved by a third annotator. The annotation guideline will be updated if needed. Finally, the IAA will be calculated on all documents excluding the first 50 documents that are used to improve the annotation and guideline. Annotation Phase: Notes: MSHS will annotate psychiatry notes only as they don’t have access to primary care notes right now. In the annotation cohort, a single patient will have only one note. Entity attribute “negation” will be used only for “social isolation loneliness” category. Notes General guidelines for annotation Annotators will annotate all mentions in a clinical note that indicate the presence (or absence) of present/past SI and SS. There may be multiple such mentions contained within a single document. The goal is to make an aggregate determination based on the various annotations within the document. The overall annotation schema is given in Table S1. The annotators will annotate all terms and phrases mentioned in Table S3 that indicate an individual’s social status based on the annotation schema. A single document may contain multiple entities from both SI and SS. Further, the annotators will annotate at the document/note level based on entities and their context available in that specific note. There are two tags for annotating at the document level and these are mentioned in Table S1. Instructions for annotation Annotations will be performed using a BRAT annotation tool made available at individual sites. This is a browser-based tool that users may access online in a secure environment. Each user will be provided a username and password to access the tool. Before starting the actual annotation process, each user will participate in an orientation session to familiarize themselves with the BRAT tool. Within the tool, annotators will be granted access to 300 [MS] and 225 [WCM] clinical notes provided as .txt files. After opening a .txt file, annotators will highlight spans of pertinent text and assign tags based on the entities in Table 1. In some cases, the highlighted terms may be used in an unrelated context (e.g., templated information: social isolation []) and can be tagged with ‘no context.’ We plan to annotate social isolation and social support at a fine-grained entity level. There are five (5) categories for social isolation: “loneliness,” “no social network,” “no emotional support,” “no instrumental support,” and “social isolation general.” Similarly, social support has four (4) categories: “social network,” “emotional support,” “instrumental support,” and “general.” Each term can be annotated with temporal information (present or past) and with negation (if a negation term precedes or follows the highlighted key terms [only for loneliness]). There are two (2) categories for document/note level annotation and these are “social isolation” and “social support” with “yes” and “no”. The document/note level annotations are provided based on the entity level annotations and the rules are provided in Table S2. Table S3 shows the lists of key terms and phrases that we have used for the development of NLP algorithm. We also identified some key terms and phrases that were excluded while identifying the social support or isolation from clinical encounter notes (See Table S4). no social networ no emotional support no instrumental support “social isolation general.” “social network,” “emotional support,” “instrumental support, “general.” “social isolation” “social support” “yes” “no” Table S3: List of typical examples of terms and phrases in the clinical encounter notes that would indicate a patient’s social support and isolation status. Table S3: Table S4: List of typical examples of exclusion terms and phrases in the clinical notes. Table S4: File naming convention Documents are named in the following naming convention: <Index Numer><person_id><note_id>_<note_date>.txt <person_id> E.g.: 1 101 1001 10001 100001 2000-01-01.txt References 1. Lauren A. Lepow, Braja Gopal Patra, et al. 2021. Extracting Social Isolation Information From Psychiatric Notes in the Electronic Health Records. In AMIA 2021 Annual Symposium, San Diego, CA, USA. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv Authors:
(1) Braja Gopal Patra, Weill Cornell Medicine, New York, NY, USA and co-first authors;
(2) Lauren A. Lepow, Icahn School of Medicine at Mount Sinai, New York, NY, USA and co-first authors;
(3) Praneet Kasi Reddy Jagadeesh Kumar. Weill Cornell Medicine, New York, NY, USA;
(4) Veer Vekaria, Weill Cornell Medicine, New York, NY, USA;
(5) Mohit Manoj Sharma, Weill Cornell Medicine, New York, NY, USA;
(6) Prakash Adekkanattu, Weill Cornell Medicine, New York, NY, USA;
(7) Brian Fennessy, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(8) Gavin Hynes, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(9) Isotta Landi, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(10) Jorge A. Sanchez-Ruiz, Mayo Clinic, Rochester, MN, USA;
(11) Euijung Ryu, Mayo Clinic, Rochester, MN, USA;
(12) Joanna M. Biernacka, Mayo Clinic, Rochester, MN, USA;
(13) Girish N. Nadkarni, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(14) Ardesheer Talati, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA;
(15) Myrna Weissman, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA;
(16) Mark Olfson, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA, New York State Psychiatric Institute, New York, NY, USA, and Columbia University Irving Medical Center, New York, NY, USA;
(17) J. John Mann, Columbia University Irving Medical Center, New York, NY, USA;
(18) Alexander W. Charney, Icahn School of Medicine at Mount Sinai, New York, NY, USA;
(19) Jyotishman Pathak, Weill Cornell Medicine, New York, NY, USA. Authors: Authors: (1) Braja Gopal Patra, Weill Cornell Medicine, New York, NY, USA and co-first authors; (2) Lauren A. Lepow, Icahn School of Medicine at Mount Sinai, New York, NY, USA and co-first authors; (3) Praneet Kasi Reddy Jagadeesh Kumar. Weill Cornell Medicine, New York, NY, USA; (4) Veer Vekaria, Weill Cornell Medicine, New York, NY, USA; (5) Mohit Manoj Sharma, Weill Cornell Medicine, New York, NY, USA; (6) Prakash Adekkanattu, Weill Cornell Medicine, New York, NY, USA; (7) Brian Fennessy, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (8) Gavin Hynes, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (9) Isotta Landi, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (10) Jorge A. Sanchez-Ruiz, Mayo Clinic, Rochester, MN, USA; (11) Euijung Ryu, Mayo Clinic, Rochester, MN, USA; (12) Joanna M. Biernacka, Mayo Clinic, Rochester, MN, USA; (13) Girish N. Nadkarni, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (14) Ardesheer Talati, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA; (15) Myrna Weissman, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA and New York State Psychiatric Institute, New York, NY, USA; (16) Mark Olfson, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA, New York State Psychiatric Institute, New York, NY, USA, and Columbia University Irving Medical Center, New York, NY, USA; (17) J. John Mann, Columbia University Irving Medical Center, New York, NY, USA; (18) Alexander W. Charney, Icahn School of Medicine at Mount Sinai, New York, NY, USA; (19) Jyotishman Pathak, Weill Cornell Medicine, New York, NY, USA.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Guidelines for Annotating Social Support and Social Isolation in Clinical Notes

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Developing Rule and LLM-Based Systems to Identify Mentions of Fine-Grained Categories

Extracting Social Support and Social Isolation Information From Clinical Psychiatry Notes

How We Used EHR Data to Develop the SS/SI NLP Pipeline

Extracting Social Support and Isolation Info From Clinical Psychiatry Notes: SS and SI Categories

How We Collected Lexicons for Fine-Grained Categories of SS and SI Using an Iterative Method

This Is How We Created Gold Standard Data for Developing the NLP Pipelines

Developing Rule and LLM-Based Systems to Identify Mentions of Fine-Grained Categories

Extracting Social Support and Social Isolation Information From Clinical Psychiatry Notes

How We Used EHR Data to Develop the SS/SI NLP Pipeline

Extracting Social Support and Isolation Info From Clinical Psychiatry Notes: SS and SI Categories

How We Collected Lexicons for Fine-Grained Categories of SS and SI Using an Iterative Method

This Is How We Created Gold Standard Data for Developing the NLP Pipelines

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps