Ipilimumab and Rituximab in Treating Patients With Relapsed or Refractory B-cell Lymphoma

Inclusion Criteria:

* Previously treated, histologically confirmed cluster of differentiation (CD)20+ B cell lymphoma; bone marrow biopsies as the sole means of diagnosis are not acceptable, but they may be submitted in conjunction with nodal biopsies or extra nodal biopsies; fine needle aspirates are not acceptable
* All patients must be informed of the investigative nature of the clinical trial and give written informed consent in accordance with institutional and federal guidelines
* Able to adhere to the study visit schedule and other protocol requirements
* Karnofsky \>= 70%
* Life expectancy expected to be greater than 3 months
* Leukocytes \>= 3,000/mcL
* Absolute neutrophil count \>= 1,000/mcL
* Platelets \>= 50,000/mcL
* Total bilirubin =\< 2.0 x institutional upper limit of normal
* Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional upper limit of normal
* Serum creatinine =\< 2.0 x upper limit of normal OR calculated creatinine clearance \>= 30 ml/min/1.73 M\^2 by the modified Cockcroft and Gault formula OR creatinine clearance \>= 30 mL/min obtained from a 24-hour urine collection
* At least one measurable lesion according to international workshop lymphoma response criteria; there must be measurable lymphadenopathy to follow with serial exam and/or imaging
* All previous cancer therapy, including radiation, hormonal therapy and surgery, must have been discontinued at least 4 weeks prior to treatment in this study
* Patients must have evidence of progression of disease during or after last treatment
* Submission of original biopsy for review and verification by participating center hematopathologist
* Disease free of prior malignancies for \>= 3 years with exception of currently treated basal cell, squamous cell carcinoma of the skin, or carcinoma "in situ" of the cervix or breast

Exclusion Criteria:

* Patients who have had chemotherapy or radiotherapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events due to agents administered more than 4 weeks earlier
* Patients with a history of prior treatment with ipilimumab
* Patients with a history of prior treatment with an anti-programmed cell death (PD) 1 antibody, CD137 agonist or other immune activating therapy such as anti-CD 40 antibody are excluded unless 5 half-lives of the agent (minimum of 8 weeks) have intervened since the therapy; patients who have received prior vaccine therapy are eligible
* Patients who are receiving any other investigational agents
* Autoimmune disease: patients with a history of inflammatory bowel disease, including ulcerative colitis and Crohn's disease, are excluded from this study, as are patients with a history of symptomatic disease (e.g., rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[e.g., Wegener's granulomatosis\]); central nervous system (CNS) or motor neuropathy considered of autoimmune origin (e.g. Guillain-Barre syndrome and myasthenia gravis, multiple sclerosis)
* Patients with known immune impairment who may be unable to respond to anti-cytotoxic T-lymphocyte antigen 4 (CTLA 4) antibody
* Patients with known uncontrolled brain metastases are excluded; however, patients with stable brain disease (off corticosteroids) at least 2 weeks after completion of appropriate therapy for their brain metastases are eligible
* History of allergic reactions attributed to compounds of similar chemical or biologic composition to rituximab
* Patients on systemic corticosteroids (except for patients on stable doses of hormone replacement therapy such as hydrocortisone), or other immunosuppressants (e.g., infliximab, mycophenolate mofetil) are excluded
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
* Patients with chronic human immunodeficiency virus (HIV), hepatitis B or hepatitis C infections are excluded
* Pregnant women are excluded from this study
* HIV-positive patients on combination antiretroviral therapy are ineligible
* Rituximab within six weeks