Sorafenib Tosylate and Pembrolizumab in Treating Patients With Advanced or Metastatic Liver Cancer

Inclusion Criteria:

* Participant must have histologically or radiographically confirmed hepatocellular cancer (HCC) that is advanced or metastatic and if archival tissue is available, have archival tissue submitted for PD-L1, PD-L2 testing
* Participants with measurable disease that has progressed are eligible if prior surgery or locoregional therapy occurred \> 28 days prior to enrollment
* Have an Eastern Cooperative Oncology Group (ECOG) performance status of 0-1 (Karnofsky \>= 60%)
* Child-Pugh class-A liver function
* Absolute neutrophil count (ANC) \>= 1,500/ mcL
* Hemoglobin \>= 8.5 g/dL
* Platelets \>= 75,000/ mcL
* Total bilirubin =\< 2.0 mg/dL
* Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) =\< 5 X ULN
* Serum Creatinine \<= 1.5 upper limit of normal (ULN)or Creatinine clearance \> 50 mL/minute if serum creatinine is elevated above 1.5 X ULN
* Have measurable disease per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria present
* Ability to swallow and retain oral medication
* Participants of child-bearing potential must agree to use adequate contraceptive methods (e.g., hormonal or barrier method of birth control; abstinence) prior to study entry; should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately
* Participant must understand the investigational nature of this study and sign an Independent Ethics Committee/Institutional Review Board approved written informed consent form prior to receiving any study related procedure
* Participants with past or ongoing hepatitis C virus (HCV) infection will be eligible for the study. The treated participants must have completed their treatment at least 1 month prior to starting study intervention.
* Participants with controlled hepatitis B will be eligible as long as they meet the following criteria:

Antiviral therapy for HBV must be given for at least 12 weeks and HBV viral load must be less than 100 IU/ml prior to first dose of study drug. Participants on active HBV therapy with viral loads under 100 IU/mL should stay on the same therapy throughout study treatment Participants who are anti-HBc , negative for Hepatitis B surface antigen (HBsAg) and negative or positive for anti-HBs, and who have an HBV viral load under 100 IU/mL , do not require HBV anti-viral prophylaxis

Exclusion Criteria:

* One prior line of therapy that may include a PDL1 blocker allowed, no prior sorafenib or PD1 blocker allowed.
* Participants who have had radiotherapy or chemotherapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events due to agents administered more than 4 weeks earlier
* Any evidence of bleeding diathesis (patients on therapeutic warfarin or heparin will be excluded)
* Participants with a history of variceal bleed within 6 months prior to enrollment
* Known human immunodeficiency virus (HIV)-positive participants (even if on combination retrovirals, participant will be excluded
* Participants with chronic autoimmune disease
* Participants with known brain metastases should be excluded from this clinical trial
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
* Has known history of, or any evidence of active, non-infectious pneumonitis
* Pregnant or nursing female participants
* Unwilling or unable to follow protocol requirements
* Any condition which in the Investigator's opinion deems the participant an unsuitable candidate to receive study drug
* Received a live vaccine within 30 days prior to start of study treatment