Pembrolizumab and Chemoradiotherapy for the Treatment of Unresectable Gastroesophageal Cancer

Inclusion Criteria:

* Participants with histologically confirmed diagnosis of unresectable gastroesophageal adenocarcinoma will be enrolled in this study for systemic treatment first
* Have histologically documented locally advanced unresectable cancer or localized cancer in a patient who declines surgery
* Participant is able to provide endoscopic research biopsies and research blood
* Participant has an Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1, as assessed within 7 days prior to treatment initiation
* A male participant must agree to use a contraception during the treatment period and for at least 120 days, corresponding to time needed to eliminate any study treatment(s) after the last dose of study treatment and refrain from donating sperm during this period
* A female participant is eligible to participate if she is not pregnant, not breastfeeding, and at least one of the following conditions applies:

  * Not a woman of childbearing potential (WOCBP) OR
  * A WOCBP who agrees to follow the contraceptive guidance during the treatment period and for at least 120 days/weeks after the last dose of study treatment
* Patients with stable brain metastasis and/or carcinomatous leptomeningeal disease as defined in exclusion criteria
* The participant (or legally acceptable representative if applicable) provides written informed consent for the trial. The participant may also provide consent for future biomedical research. However, the participant may participate in the main trial without participating in future biomedical research
* Absolute neutrophil count \>= 1500/uL (within 10 days prior to the start of study treatment)
* Platelets \>= 100 000/uL (within 10 days prior to the start of study treatment)
* Hemoglobin \>= 9.0 g/dL or \>= 5.6 mmol/L (within 10 days prior to the start of study treatment)

  * Criteria may be met with blood transfusion as these tumors are losing blood
* Creatinine =\< 1.5 x upper limit of normal (ULN) OR measured or calculated\* creatinine clearance (CrCl) \>= 60 mL/min for participant with creatinine levels \> 1.5 x institutional ULN (within 10 days prior to the start of study treatment) (GFR can also be used in place of creatinine or CrCl)

  * Creatinine clearance (CrCl) should be calculated per institutional standard
* Total bilirubin =\<1.5 x ULN OR direct bilirubin =\< ULN for participants with total bilirubin levels \> 1.5 x ULN (within 10 days prior to the start of study treatment)
* Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x ULN (within 10 days prior to the start of study treatment)
* International normalized ratio (INR) OR prothrombin time (PT), activated partial thromboplastin time (aPTT) =\< 1.5 x ULN unless participant is receiving anticoagulant therapy as long as PT or aPTT is within therapeutic range of intended use of anticoagulants (within 10 days prior to the start of study treatment)

Exclusion Criteria:

* Has cancer that is confined to the stomach and not involving gastroesophageal junction
* Has significant cardiovascular impairment within 6 months of the first dose of study drug (New York Heart Association \[NYHA\] class III or IV)
* Has had major surgery, open biopsy, or significant traumatic injury within 28 days prior to randomization, or anticipation of the need for major surgery during the course of study treatment

  * Note: If participant has had major surgery, they must have recovered adequately from the toxicity and/or complications from the treatment prior to starting study intervention
* Has pre-existing peripheral neuropathy \> grade 1
* Participant has a history of (non-infectious) pneumonitis that required treatment with steroids or currently has pneumonitis
* Participant has a known additional malignancy that is progressing or has required active treatment in the last 2 years

  * Note: Participants with basal cell carcinoma of the skin, squamous cell carcinoma of the skin, ductal carcinoma in situ, or cervical carcinoma in situ that has undergone potentially curative therapy are not excluded
* Participant has known active central nervous system (CNS) metastases and/or carcinomatous meningitis

  * Note: Participants with previously treated brain metastases may participate provided they are radiologically stable (i.e., without evidence of progression for at least 4 weeks by repeat imaging \[repeat imaging should be performed during study screening\]), clinically stable, and without requirement for steroid treatment for at least 14 days prior to the first dose of study treatment
* Participant has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of study treatment
* Participant has an active autoimmune disease that has required systemic treatment in the past 2 years (i.e., with use of disease modifying agents, corticosteroids or immunosuppressive drugs)

  * Note: Replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency) is not considered a form of systemic treatment
* Participant has an active infection requiring systemic therapy
* Participant has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with the participant's involvement for the full duration of the trial, or is not in the best interest of the participant to be involved, in the opinion of the treating investigator

  * Note: Participants who received colony-stimulating factors (e.g., granulocyte colony stimulating factor \[G-CSF\], granulocyte macrophage colony-stimulating factor \[GM-CSF\] or recombinant erythropoietin) within 2 weeks prior to the first dose of study treatment are not eligible
* Participant has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial
* Participant has a known history of human immunodeficiency virus (HIV) infection. No HIV testing is required unless mandated by local health authority
* Participant has a known history of hepatitis B (defined as hepatitis B surface antigen \[HBsAg\] reactive) or known active hepatitis C virus (defined as HCV ribonucleic acid \[RNA\] \[qualitative\] is detected) infection. No testing for hepatitis B or hepatitis C is required
* Participant has received prior therapy with an anti-PD-1, anti-PD-L1, or anti-PD-L2 agent or with an agent directed to another stimulatory or co-inhibitory T cell receptor (e.g., CTLA-4, OX 40, CD137)
* Participant has received prior systemic anti-cancer therapy including investigational agents
* Has received prior radiotherapy for the current disease
* Participant has received a live vaccine within 30 days prior to the first dose of study treatment. Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, varicella/zoster (chicken pox), yellow fever, rabies, Bacillus Calmette-Guerin (BCG), and typhoid vaccine. Seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., FluMist) are live attenuated vaccines and are not allowed
* Participant has severe hypersensitivity (\>= grade 3) to pembrolizumab or any of its excipients
* Participant is currently participating in or has participated in a study of an investigational agent (anti-cancer agent only) or has used an investigational device within 4 weeks prior to the first dose of study treatment

  * Note: Participants who have entered the follow-up phase of an investigational trial may participate as long as it has been 4 weeks after the last dose of the previous investigational agent
* Participant is pregnant or breastfeeding or expecting to conceive children within the projected duration of the study, starting with the screening visit through 120 days after the last dose of study treatment
* Participant has symptomatic ascites or pleural effusion. A participant who is clinically stable following treatment for these conditions (including therapeutic thoraco- or paracentesis) is eligible
* Has had an allogeneic tissue/solid organ transplant