Neutron Radiation Therapy and Pembrolizumab in Treating Participants With Advanced Urothelial Carcinoma

Inclusion Criteria:

* Pathologically proven (either histologic or cytologic) diagnosis of urothelial carcinoma
* At least two sites of disease that are measurable by Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria
* Eligible for checkpoint inhibitor immunotherapy (pembrolizumab) per standard of care
* No history of autoimmune disease requiring systemic therapy (e.g. steroids or biologic agents)
* Absolute neutrophil count (ANC) ≥ 1500 /mcL
* Platelets ≥ 100,000/mcL
* Hemoglobin \> 9 g/dL
* Creatinine ≤ 1.5 x upper limit of normal (ULN) OR ≥ 60 mL/min
* Total bilirubin ≤ 1.5 ULN OR direct bilirubin ≤ ULN if total bilirubin \> 1.5 x ULN
* Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) ≤ 2.5 x ULN OR \< 5 x ULN if patient has liver metastasis
* Albumin \>= 2.5 g/dL
* International normalized ratio (INR) or PT ≤ 1.5 x ULN unless on anticoagulation therapy, in which case prothrombin time (PT) or partial thromboplastin time (PTT) should be in the therapeutic range
* PTT ≤ 1.5 x ULN unless on anticoagulation therapy, in which case PT or PTT should be in the therapeutic range
* Eligible for neutron radiation treatment to 1-3 sites of metastatic disease (lesions do not have to be symptomatic)
* No steroids for at least 2 weeks prior to enrollment, and patient must not be expected to require steroids during the study period
* Zubrod performance status 0-2
* Patient must sign study specific informed consent prior to study entry
* Patients who are sexually active must use medically acceptable forms of contraception
* Life expectancy must be \> 3 months

Exclusion Criteria:

* Has a known history of active TB (Bacillus tuberculosis)
* Hypersensitivity to pembrolizumab or any of its excipients
* Has a known additional malignancy that is progressing or requires active treatment
* Has received prior therapy with an anti-PD-1, anti-PD-L1, or anti-PD-L2 agent
* Has a known history of human immunodeficiency virus (HIV) (HIV 1/2 antibodies)
* Has known active hepatitis B (e.g., hepatitis B virus surface antigen \[HBsAg\] reactive) or hepatitis C (e.g., hepatitis C virus \[HCV\] ribonucleic acid \[RNA\] \[qualitative\] is detected)
* Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial
* Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with the subject's participation for the full duration of the trial, or is not in the best interest of the subject to participate, in the opinion of the treating investigator
* Has known history of, or any evidence of active, non-infectious pneumonitis
* Has an active infection requiring systemic therapy
* Has received a live vaccine within 30 days of planned start of study therapy
* Has known active central nervous system (CNS) metastases and/or carcinomatous meningitis; subjects with previously treated brain metastases may participate provided there is no evidence of new or enlarging brain metastases, and are not using steroids for at least 7 days prior to trial treatment; this exception does not include carcinomatous meningitis which is excluded regardless of clinical stability