Talimogene Laherparepvec and Pembrolizumab in Treating Patients With Stage III-IV Melanoma

Inclusion Criteria:

* Patients must have pathologically confirmed stage IV or unresectable stage III melanoma; patients must not have disease that is suitable for local therapy, administered with curative intent
* Patients must have measurable disease per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1; contrast-enhanced computed tomography (CT) scans of the chest, abdomen and pelvis are required; a whole body positron emission tomography (PET)/CT scan with diagnostic quality images and intravenous iodinated contrast may be used in lieu of a contrast enhanced CT of the chest, abdomen and pelvis; imaging of the head and neck, or the limbs is required only if the patient has a lesion(s) in these areas; contrast may be omitted if the treating investigator believes that exposure to contrast poses an excessive risk to the patient; if skin lesions are being followed as measurable disease, photograph with a ruler included and physician's measurements, must be kept in the patients chart as source documentation; all measurable lesions must be assessed within 28 days prior to registration; tests to assess non-measurable disease must be performed within 42 days prior to registration.; all disease must be assessed and documented on the baseline tumor assessment form (RECIST 1.1)
* Cohort A: Patients must have at least one measurable visceral lesion (per RECIST 1.1); a visceral lesion is any solid organ except for skin, lymph node, and musculoskeletal tissue; at least one of these visceral lesions must be measurable per RECIST 1.1
* Patients must, in the opinion of the treating physician, be candidates for intralesional administration into cutaneous, subcutaneous, or nodal lesions
* Patients may have brain metastases if all lesions have been treated with stereotactic radiation therapy, craniotomy, or gamma knife therapy and have not required steroids for at least 14 days prior to registration
* Patient must have had prior treatment with anti-PD-1 or anti-PD-L1 agents and have documented disease progression on these agents prior to registration; patients who have progressed after adjuvant anti-PD1/L1 agents are eligible
* Patients must be \>= 18 years of age
* Patients must have Zubrod performance status =\< 2
* Absolute neutrophil count (ANC) \>= 1,500/mcL (within 28 days prior to registration)
* Hemoglobin \>= 8 g/dL (within 28 days prior to registration)
* Platelets \>= 100,000/mcL (within 28 days prior to registration)
* Albumin \>= 2.5 g/dL (within 28 days prior to registration)
* Total bilirubin =\< 1.5 x institutional upper limit of normal (IULN) except patients with documented Gilbert's syndrome (=\< 3 x IULN is eligible) (within 28 days prior to registration)
* Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) both =\< 3 x IULN (within 28 days prior to registration)
* Patients must have lactate dehydrogenase (LDH) obtained prior to registration
* Patients must have complete physical examination and medical history obtained within 28 days prior to registration
* Patients must be offered the opportunity to submit archival tissue for translational medicine; patients must also be willing to undergo biopsies and submit tissue and blood for translational medicine; with patients consent, any remaining specimens will be banked for future use
* Patients must be informed of the investigational nature of this study and must sign and give written informed consent in accordance with institutional and federal guidelines
* As a part of the Oncology Patient Enrollment Network (OPEN) registration process the treating institution's identity is provided in order to ensure that the current (within 365 days) date of institutional review board approval for this study has been entered in the system

Exclusion Criteria:

* Cohort B: Patients must not have any visceral lesions
* Patients must not have had surgery, biologic therapy, or hormonal therapy within 14 days prior to registration; patients must not have had chemotherapy, targeted small molecule therapy, or radiation therapy within 14 days prior to registration; patients must not have had a monoclonal antibody for cancer treatment, except anti-PD1/L1 antibodies, within 28 days prior to registration

  * Patients must have recovered from all adverse events due to prior anti-cancer therapy (residual toxicity =\< grade 1) prior to registration, with the exception of patients with =\< grade 2 neuropathy, =\< grade 2 hypothyroidism, or =\< grade 2 alopecia
  * If patients received major surgery, they must have recovered adequately from toxicity and/or complications from the intervention prior to registration
* Patients must not have received prior treatment with talimogene laherparepvec (T-VEC); prior treatment with T-VEC is defined as receiving at least one injection with 1 x 10\^8 plaque forming units (pfu)
* Patients must not have received any live vaccine within 30 days prior to registration; seasonal flu vaccines that do not contain live virus are permitted
* Patients must not be planning to receive other biologic therapy, radiation therapy, hormonal therapy, chemotherapy, surgery, or other therapy while on this protocol; palliative radiation therapy or surgery can be considered for symptomatic non-target lesions after discussions with the study team
* Patients must not require use of systemic corticosteroid within 14 days prior to registration or during protocol treatment; patients with preexisting severe autoimmune disease requiring systemic corticosteroids or ongoing immunosuppression are not eligible
* Patients must not have known history of hepatitis B, hepatitis C, or human immunodeficiency virus (HIV) due to contraindication of talimogene laherparepvec (T-VEC) in immune-compromised patients and that administration of talimogene laherparepvec (T-VEC) has not been tested in HIV-positive patients; the use of physiologic doses of corticosteroids may be approved after consultation with the study chair
* Patients must not have history of (non-infectious) pneumonitis that required steroids or current pneumonitis
* Patients must not have an active infection requiring systemic therapy nor a viral infection requiring intermittent treatment with an antiherpetic drug, other than intermittent topical use
* Patients must not have active herpetic skin lesions or prior complications of herpetic infection (e.g., herpetic keratitis or encephalitis) which requires intermittent or chronic treatment with an anti-herpetic drug other than intermittent topical use
* Patients must not have organ allografts
* Patients must not have an uncontrolled intercurrent illness or whose control may be jeopardized by the treatment with the study therapy, or psychiatric illness/social situations which would limit compliance with study requirements
* Patients must not have active autoimmune disease (e.g., pneumonitis, glomerulonephritis, vasculitis, or other) that requires systemic treatment (i.e., use of corticosteroids, immunosuppressive drugs or biological agents used for treatment of autoimmune diseases) in the past 2 years; replacement therapy (e.g., thyroxine for hypothyroidism, insulin for diabetes or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency) is not considered a form of systemic treatment for autoimmune disease
* Patient must not have evidence of any clinically significant immunosuppression such as the following:

  * Primary immunodeficiency state such as severe combined immunodeficiency disease;
  * Concurrent opportunistic infection;
  * Receiving systemic immunosuppressive therapy (\> 2 weeks) including oral steroid doses \> 10 mg/day of prednisone or equivalent within 2 months prior to enrollment
* Patients must not have any other malignancy that requires active treatment
* Patients must not be pregnant or nursing due to risk of fetal or nursing infant harm; women of reproductive potential must have a negative serum pregnancy test within 7 days prior to registration; women/men of reproductive potential must have agreed to use an effective contraceptive method while on study and for 120 days after last study treatment; a woman is considered to be of "reproductive potential" if she has had menses at any time in the preceding 12 consecutive months; in addition to routine contraceptive methods, "effective contraception" also includes heterosexual celibacy and surgery intended to prevent pregnancy (or with a side-effect of pregnancy prevention) defined as a hysterectomy, bilateral oophorectomy or bilateral tubal ligation; however, if at any point a previously celibate patient chooses to become heterosexually active during the time period for use of contraceptive measures outlined in the protocol, he/she is responsible for beginning contraceptive measures