Testing the Addition of Radiation Therapy to Immunotherapy for Merkel Cell Carcinoma

Inclusion Criteria:

* Patients must have pathologically (histologically or cytologically) proven diagnosis of MCC by local pathology review
* Have measurable disease based on RECIST 1.1 including at least two cancerous deposits; at least one deposit must be RECIST measurable while at least one deposit must meet criteria for SBRT; non-radiated tumor will be identified prior to randomization on the protocol
* Patients must have advanced or metastatic MCC defined as evidence of distant metastasis(es) on imaging

  * Patients with locoregionally confined disease are not eligible
* No prior immunotherapy for advanced/metastatic MCC
* Patients with known or suspected central nervous system (CNS) metastases, untreated CNS metastases, or with the CNS as the only site of disease are excluded; however, subjects with controlled brain metastases will be allowed to enroll; controlled brain metastases are defined as no radiographic progression for at least 4 weeks following radiation and/or surgical treatment (or 4 weeks of observation if no intervention is clinically indicated), and off of steroids for at least 2 weeks, and no new or progressive neurological signs and symptoms
* Patients having received palliative radiotherapy for extracranial metastasis(es) are eligible as long as there are 2 cancerous deposits that have not received prior radiation therapy (RT) and they meet the following criteria

  * No prior radiation therapy (\> 5 Gy) to the metastasis intended to be treated with SBRT
* No history of the following:

  * Autoimmunity requiring systemic immunosuppression within 2 years
  * Patients known to be human immunodeficiency virus (HIV) positive are eligible if they meet the following:

    * CD4 counts \>= 350 mm\^3
    * Serum HIV viral load of \< 25,000 IU/ml
* No other active malignancy that the investigator determines would interfere with the treatment and safety analysis
* Not pregnant and not nursing, because this study involves an investigational agent whose genotoxic, mutagenic and teratogenic effects on the developing fetus and newborn are unknown; therefore, for women of childbearing potential only, a negative (if your test schedule specifically indicates a urine or serum pregnancy test, add that information at this point) pregnancy test done =\< 28 days prior to registration is required
* Eastern Cooperative Oncology Group (ECOG) performance status 0-2
* Absolute neutrophil count (ANC) \>= 1,500/mm\^3
* Platelet count \>= 100,000/mm\^3
* Hemoglobin \>= 9.0 g/dl
* Total bilirubin =\< 2.0 mg/dl
* Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 3.0 x upper limit of normal (ULN)
* Systolic blood pressure (BP) =\< 150 mg HG
* Diastolic BP =\< 90 mg HG
* Albumin \> 3 mg/dl
* Blood urea nitrogen (BUN) =\< 30 mg/dl
* Creatinine =\< 1.7 mg/dl
* The following imaging workup to document metastases within 45 days prior to study registration are required: CT scans of the chest, abdomen and pelvis with radionuclide bone scan OR whole body (at least skull base to midthigh) positron emission tomography (PET)/CT