Brentuximab Vedotin and Nivolumab for the Treatment of Patients With Relapsed/Refractory Classical Hodgkin Lymphoma

Inclusion Criteria:

* Documented informed consent of the participant and/or legally authorized representative

  * Assent, when appropriate, will be obtained per institutional guidelines
* Be willing to provide tissue (either from a fresh core or excisional biopsy performed as standard of care, or from archival tissue) of a biopsy that was performed after frontline systemic therapy, and prior to starting protocol therapy

  * If unavailable, exceptions may be granted with study principal investigator (PI) approval
* Eastern Cooperative Oncology Group (ECOG) =\< 2
* Histologically confirmed diagnosis of classical Hodgkin lymphoma (excluding nodular lymphocyte predominant Hodgkin lymphoma) according to the World Health Organization (WHO) classification, with hematopathology review at the participating institution
* Relapsed or refractory disease after no more than 1 line of prior therapy (not counting radiotherapy). However, a maximum of 5 patients with primary refractory disease may be enrolled in this study.

Note: Patients who received BV or an anti-PD-1/PD-L1 agent as part of frontline therapy are eligible if they achieved a CMR with frontline therapy and have not relapsed within 6 months from the end of frontline therapy Relapse must have been confirmed histologically (with hematopathology review at the participating institution)

* Not a candidate for ASCT, based on age, co-morbidities, or patient preference. The reason for ASCT non-candidacy must be documented in the Case Report Form and verified by the site PI
* Measurable disease (at least one non-bony fludeoxyglucose F-18 \[FDG\]-avid lesion \>= 1.5 cm in long axis)
* Absolute neutrophil count (ANC) \>= 1,000/mm\^3

  * NOTE: Growth factor is not permitted within 7 days of ANC assessment unless cytopenia is secondary to disease involvement
* Platelets \>= 50,000/mm\^3

  * NOTE: Platelet transfusions are not permitted within 7 days of platelet assessment unless cytopenia is secondary to disease involvement
* Hemoglobin \>= 8 g/dL (no transfusion allowed within 3 days prior to screening)
* Total bilirubin =\< 1.5 x upper limit of normal (ULN) or direct bilirubin =\< 1.5 x ULN for patients with Gilbert's disease
* Aspartate aminotransferase (AST) =\< 2.5 x ULN
* Alanine aminotransferase (ALT) =\< 2.5 x ULN
* Creatinine clearance of \>= 40 mL/min per 24 hour urine test or the Cockcroft-Gault formula
* Women of childbearing potential (WOCBP): negative urine or serum pregnancy test. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required
* Agreement by women and men of childbearing potential\* to use an effective method of birth control or abstain from heterosexual activity for the course of the study through at least 5 months (women) or 7 months (men) after the last dose of protocol therapy

  * Childbearing potential defined as not being surgically sterilized (men and women) or have not been free from menses for \> 1 year (women only)

Exclusion Criteria:

* Concomitant investigational therapy
* Live vaccine within 30 days prior to day 1 of protocol therapy (e.g. measles, mumps, rubella, varicella, yellow fever, rabies, bacillus Calmette-Guerin \[BCG\], oral polio vaccine, and oral typhoid)
* Grade \>= 2 peripheral neuropathy
* History of prior \>= grade 3 hypersensitivity to either brentuximab vedotin or nivolumab
* Known active central nervous system (CNS) involvement by lymphoma, including parenchymal and/or lymphomatous meningitis
* History of another primary malignancy that has not been in remission for at least 3 years, with the following exceptions:

  * Non-melanoma skin cancer treated with curative intent
  * In situ cervical cancer
  * If the malignancy is expected to not require any treatment for at least 2 years (this exception should be discussed with the study PI)
* Condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone or equivalent) or other immunosuppressive medications within 14 days of study drug administration. Exceptions are:

  * Inhaled or topical steroids and
  * Adrenal replacement doses \> 10 mg daily prednisone equivalents in the absence of active autoimmune disease
* History of progressive multifocal leukoencephalopathy (PML)
* Prior diagnosis of inherited or acquired immunodeficiency
* Active pneumonitis or interstitial lung disease
* Active, known or suspected autoimmune disease. The following are exceptions:

  * Vitiligo
  * Psoriasis not requiring systemic treatment
  * Hemolytic anemia associated with the lymphoma
  * Type I diabetes mellitus, if adequately controlled with therapy
  * Thyroid disease, if adequately controlled with therapy
  * Conditions not expected to recur in the absence of an external trigger (such exceptions should be discussed with the study PI)
* Active history of:

  * Hepatitis B (hepatitis B virus \[HBV\]) or C (hepatitis C virus \[HCV\]) infection. Patients with past HBV infection (defined as negative hepatitis B surface antigen \[HBsAg\] and positive hepatitis B core antibody \[HBcAb\]) are eligible if HBV DNA is undetectable. Patients who are positive for HCV antibody are eligible if polymerase chain reaction (PCR) is negative for HCV ribonucleic acid (RNA)
  * Human immunodeficiency virus (HIV) infection. Subjects who have an undetectable or unquantifiable human immunodeficiency virus (HIV) viral load with CD4 \>= 200 and are on highly active antiretroviral therapy (HAART) medication are allowed. Testing to be done only in patients suspected of having infections or exposures
* History of a cerebral vascular event (stroke or transient ischemic attack), unstable angina, myocardial infarction, or cardiac symptoms consistent with New York Heart Association class III-IV within 6 months prior to day 1 of protocol therapy
* Pregnant or breastfeeding
* Any other condition that would, in the Investigator's judgment, contraindicate the patient's participation in the clinical study due to safety concerns with clinical study procedures
* Prospective participants who, in the opinion of the investigator, may not be able to comply with all study procedures (including compliance issues related to feasibility/logistics)