Talimogene Laherparepvec and Panitumumab for the Treatment of Locally Advanced or Metastatic Squamous Cell Carcinoma of the Skin

Inclusion Criteria:

* Histologically confirmed squamous cell carcinoma of the skin (SCCS) that is a) locally advanced or metastatic for which curative surgery or radiation would be difficult or impossible, or b) recurrent after initial surgery, chemotherapy, or radiation therapy, or c) considered to have aggressive features including the following: tumors 2 cm or more, tumors invading deep tissues such as muscle, cartilage or bone; tumors showing perineural invasion, and/or tumors metastatic to loco-regional lymph nodes. Patients may have had prior surgical interventions or been treated with investigational agents with residual or recurrent disease
* Tumor suitable for direct or ultrasound-guided injection defined as at least one cutaneous, subcutaneous, or nodal lesion, or aggregate of lesions, \>= 10 mm in diameter
* Eastern Cooperative Oncology Group (ECOG) performance status =\< 2
* No prior treatment with panitumumab or talimogene laherparepvec for advanced disease
* Prior surgery or radiation is allowed if there is documented progression in the radiated/resected area or elsewhere by Response Evaluation Criteria in Solid Tumors (RECIST) criteria version (v) 1.1
* Measurable disease by RECIST criteria v 1.1
* Patients with a history of hematologic or solid organ transplant will be considered if they do not require high dose steroids or high dose immunosupressants for disease control or control of transplant rejection, and have adequate hematologic, renal, and hepatic function as specified below. Current medications must be reviewed with transplant pharmacy team to exclude potentially serious interactions and case discussed with the study principal investigator (PI)
* Second primary malignancy only if treatment would interfere with the patient?s participation in this trial in the opinion of the treating physician. Clear exceptions are 1) patient had a second primary malignancy but has been treated and disease free for at least 3 years, 2) in situ carcinoma (e.g., in situ carcinoma of the cervix) and, 3) additional skin cancers that have been definitively treated by surgery and/or radiation. Patients with chronic lymphocytic leukemia will be allowed if their blood counts are within acceptable hematologic parameters and if they are not currently requiring cytotoxic or biologic anticancer treatment (supportive treatment such as intravenous immunoglobulin \[IVIG\] is permitted)
* Patients with autoimmune disorders will be considered if they do not require high dose steroids or other immunosuppressants for disease control. Prednisone in daily doses up to 10 mg and inhaled steroids are acceptable
* Absolute neutrophil count (ANC) \>= 1500/uL
* Platelet count \>= 100,000/mm\^2
* Hemoglobin \>= 9 g/dL
* Total bilirubin \< 1.5 x institutional upper limit of normal (ULN); if patient has conditions of congenital hyperbilirubinemia, then patient must have isolated hyperbilirubinemia (e.g., no other liver function test abnormalities) with maximum bilirubin \< 2 x institutional ULN
* Aspartate aminotransferase (AST) or alanine aminotransferase (ALT) =\< 2.5 x institutional ULN in absence of liver metastases; =\< 5 x ULN in presence of liver metastases
* Alkaline phosphatase \< 2.5 x institutional ULN
* Creatinine \< 1.5 x institutional ULN or calculated creatinine clearance \>= 60 mL/min as estimated using the Cockcroft-Gault formula

Exclusion Criteria:

* Pregnant women. Women of childbearing age must be willing to undergo a pregnancy test prior to therapy and to use adequate contraception (e.g., hormonal or barrier method of contraception or abstinence) for the duration of the study and 6 months thereafter. Should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately. Menopausal status will be defined as one or more of successful hysterectomy, bilateral tubal ligation or bilateral oophorectomy, amenorrhea \>= 12 consecutive months without another cause, or a documented serum follicle stimulating hormone (FSH) \>= 35 mIU/mL
* Tumor not suitable for direct or ultrasound-guided injection
* Prior treatment with talimogene laherparepvec for advanced disease
* Patients with active, uncontrolled infections including active herpetic infections or chronic herpetic infections requiring anti-viral therapy (e.g., acyclovir)
* Patients without adequate organ function as documented above
* History of allergic reactions attributed to compounds of similar chemical or biologic composition to panitumumab, talimogene laherparepvec or other agents used in the study
* History of interstitial pneumonitis, pulmonary fibrosis, or evidence of interstitial pneumonitis