Pembrolizumab Alone or With Idelalisib or Ibrutinib in Treating Patients With Relapsed or Refractory Chronic Lymphocytic Leukemia or Other Low-Grade B-Cell Non-Hodgkin Lymphomas

Inclusion Criteria:

* CLL/SLL PATIENTS (ARM A) ONLY
* Diagnosis of CLL according to the National Cancer Institute (NCI) criteria or SLL according to the World Health Organization (WHO) criteria; this includes previous documentation of:

  * Biopsy-proven small lymphocytic lymphoma or
  * Diagnosis of CLL according to NCI working group criteria as evidenced by all of the following:

    * Peripheral blood B cell count of \> 5 x 10\^9/L consisting of small to moderate size lymphocytes
    * Immunophenotyping consistent with CLL defined as:

      * The predominant population of lymphocytes share both B-cell antigens (cluster of differentiation \[CD\]19, CD20 \[typically dim expression\] or CD23) as well as CD5 in the absence of other pan-T-cell markers (CD3, CD2, etc.)
      * Clonality as evidenced by kappa or lambda light chain expression (typically dim immunoglobulin expression) or other genetic method (e.g. immunoglobulin heavy chain variable \[IGHV\] analysis)
      * NOTE: splenomegaly, hepatomegaly, or lymphadenopathy are not required for the diagnosis of CLL
    * Before diagnosing CLL or SLL, mantle cell lymphoma must be excluded by demonstrating a negative fluorescent in situ hybridization (FISH) analysis for t(11;14) (immunoglobulin H \[IgH\]/cyclin D1 \[CCND1\]) on peripheral blood or tissue biopsy or negative immunohistochemical stains for cyclin D1 on involved tissue biopsy
* Patients must be previously treated with at least one prior line of therapy; EXCEPTION: CLL patients with Richter's transformation or Hodgkin transformation do not need prior therapy to enroll

  * NOTE:

    * Prior chemotherapy or biologic novel therapy or anti-cancer monoclonal antibody based therapy for treatment of CLL will be considered prior therapy; nutraceutical treatments with no established benefit in CLL (such as epigallocatechin gallate or EGCG, found in green tea or other herbal treatments) will not be considered "prior treatment"
    * Prior oral corticosteroid therapy for an indication other than CLL will not be considered "prior treatment"
    * Previous use of corticosteroids in the combination with other therapy for treatment of autoimmune complications of CLL does constitute prior therapy for CLL
* CLL/SLL patients must have progressive disease with any one of the following characteristics based on standard criteria for treatment as defined by the NCI-Working Group (WG) 1996

  * Symptomatic CLL characterized by any one of the following:

    * Weight loss \>= 10% within the previous 6 months
    * Extreme fatigue attributed to CLL
    * Fevers \>= 100.5 degree Fahrenheit (F) for 2 weeks without evidence of infection
    * Drenching night sweats without evidence of infection
  * Evidence of progressive bone marrow failure with hemoglobin =\< 11 g/dL or platelet count =\< 100 x 10\^9/L
  * Symptomatic or progressive lymphadenopathy, splenomegaly, or hepatomegaly
  * Note: marked hypogammaglobulinemia or the development of a monoclonal protein in the absence of any of the above criteria for active disease are not sufficient for protocol therapy OR biopsy proven Richter's transformation or Hodgkin transformation of the CLL; NOTE: both untreated and previously treated patients in this category can be enrolled; they do not need to meet the progressive disease criteria in first bullet as long as measurable disease can be detected by positron emission tomography (PET)/computed tomography (CT) or CT (\>= 1.5 cm in diameter)
* LOW GRADE B-NHL PATIENTS ONLY
* Histologically confirmed relapsed (response to last treatment \>= 6 months duration) or refractory (no response to last treatment or response duration \< 6 months) indolent/low grade B cell NHL; NOTE: if patient has received previous anti-PD-1 or anti-PDL-1 consult with study chair

  * Follicular lymphoma, grades 1, 2 and 3
  * Extranodal marginal zone B-cell lymphoma of mucosa-associated lymphoid tissue (MALT) type
  * Splenic and nodal marginal zone lymphoma
  * Lymphoplasmacytic lymphoma including Waldenstrom macroglobulinemia
* Measurable disease (at least 1 lesion of \>= 1.5 cm in diameter) as detected by CT or the CT images of the PET/CT; NOTE: patients with Waldenstrom macroglobulinemia are not required to have measurable disease by CT or PET/CT if monoclonal protein is detectable by serum protein electrophoresis and/or immunoglobulin M (IgM) level is at least 2 times upper limit of normal
* CLL WITH RICHTER's TRANSFORMATION (ARM C) ONLY
* CLL diagnosis confirmed as have biopsy-proven Richter's transformation; NOTE: both untreated and previously treated patients in this category can be enrolled as long as measurable disease can be detected by PET/CT or CT (\>= 1.5 cm in diameter)
* ALL PATIENTS
* Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0 or 1
* Creatinine =\< 1.5 x upper limit of normal (ULN) OR creatinine clearance \>= 60 mL/min for subject with creatinine levels \> 1.5 x institutional ULN (obtained =\< 14 days prior to registration)
* Platelet count \>= 25 x 10\^9/L (obtained =\< 14 days prior to registration)
* Absolute neutrophil count \>= 0.5 x 10\^9/L (obtained =\< 14 days prior to registration)
* Total bilirubin =\< 1.5 x upper limit of normal (ULN) unless due to Gilbert's disease; if total bilirubin is \> 1.5 x ULN, a direct bilirubin should be performed and must be =\< upper limit of normal (obtained =\< 14 days prior to registration)
* Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) or alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 X ULN (obtained =\< 14 days prior to registration)
* Negative pregnancy test done =\< 7 days prior to registration, for women of childbearing potential only
* Provide informed written consent
* Willing to return to enrolling institution for follow-up (during the Active Monitoring Phase of the study)

  * Note: During the Active Monitoring Phase of a study (i.e., active treatment and observation), participants must be willing to return to the consenting institution for follow-up
* Willing to provide bone marrow, tissue, and blood samples for correlative research purposes
* Must have failed or be unable to tolerate or refused other available Food and Drug Administration (FDA) approved effective therapies; NOTE: patients should not have other treatment options considered curative

Exclusion Criteria:

* Currently participating in or has participated in a study of an investigational agent or using an investigational device =\< 28 days prior to registration
* Receiving systemic steroid therapy or any other form of systemic immunosuppressive therapy =\< 7 days prior to registration; EXCEPTIONS:

  * Low doses of steroids (=\< 20 mg of prednisone or equivalent dose of other steroid/day)
  * Previous use of corticosteroids is allowed
  * After initiation of MK-3475 therapy, steroid can be used for management of potential immune mediated adverse events (AE) for less than 8 weeks of therapy
  * Topical, ocular, intra-articular, intranasal, and inhalational corticosteroids (with minimal systemic absorption) are permitted
* Prior anti-cancer monoclonal antibody =\< 28 days prior to registration or who has not recovered (i.e., =\< grade 1 or at baseline) from adverse events due to agents administered more than 4 weeks earlier
* Prior chemotherapy or radiation therapy =\< 14 days prior to registration or who has not recovered (i.e., =\< grade 1 or at baseline) from adverse events due to a previously administered agent

  * Note: subjects with =\< grade 2 neuropathy are an exception to this criterion and may qualify for the study
  * Note: if subject received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention prior to starting therapy
* Known additional malignancy that is progressing or requires active treatment; EXCEPTIONS (these following exceptions are permitted to enroll in this trial):

  * Basal cell carcinoma or squamous cell carcinoma or melanoma of the skin that has undergone or will undergo potentially curative therapy
  * In situ cervical cancer that has undergone or will undergo potentially curative therapy
* Active autoimmune disease requiring systemic treatment within the past 3 months or a documented history of clinically severe autoimmune disease/syndrome difficult to control in the past; EXCEPTIONS:

  * Subjects with vitiligo or resolved childhood asthma/atopy would be an exception to this rule
  * Subjects that require intermittent use of bronchodilators or local steroid injections would not be excluded from the study
  * Subjects with hypothyroidism stable on hormone replacement, diabetes or Sjogren's syndrome are permitted for the study
  * Patients who have a positive Coombs test but no evidence of hemolysis are permitted for participation
  * Patients with psoriasis not requiring systemic treatment are permitted for participation
  * Conditions not expected to recur in the absence of an external trigger are permitted to enroll
* Evidence of interstitial lung disease or active, non-infectious pneumonitis
* Active infection requiring systemic therapy; NOTE: when the infection is controlled, patients are permitted for this study
* Known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial
* Any of the following because this study involves an investigational agent whose genotoxic, mutagenic, and teratogenic effects on the developing fetus and newborn are unknown:

  * Pregnant women
  * Nursing women
  * Men or women of childbearing potential who are unwilling to employ adequate contraception starting with the pre-screening or screening visit through 120 days after the last dose of trial treatment
* Known to be human immunodeficiency virus (HIV) positive
* Known active hepatitis B (e.g., hepatitis B surface antigen \[HBsAg\] reactive) or hepatitis C (e.g., hepatitis C virus \[HCV\] ribonucleic acid \[RNA\] \[qualitative\] is detected); NOTE: patients with active hepatitis B defined by hepatitis B surface antigen positivity or core antibody positivity in the presence of hepatitis B deoxyribonucleic acid (DNA) are not eligible for this study; patients with a positive hepatitis B core antibody but with negative hepatitis B DNA may participate, but must have hepatitis serologies and hepatitis B DNA monitored periodically by the treating physician

  * NOTE: intravenous immunoglobulin (IVIG) can cause a false positive hepatitis B serology; if patients receiving routine IVIG have core antibody or surface antigen positivity without evidence of active viremia (negative hepatitis B DNA) they may still participate in the study, but should have hepatitis serologies and hepatitis B DNA monitored periodically by the treating physician
* Received a live vaccine =\< 30 days prior to registration
* New York Heart Association classification III or IV cardiovascular disease or recent myocardial infarction or unstable angina pectoris or cardiac arrhythmia (\< 30 days)
* Active central nervous system (CNS) lymphoma or cerebrospinal fluid involvement with malignant lymphoma cells that requires therapy
* Has a clinically significant coagulopathy per investigator's assessment
* Has received an allogeneic stem cell transplant
* CLL ARMS (ARM A and ARM C) FOR COMBINATION THERAPY INCLUDING IBRUTINIB or IDELALISIB:
* Is chronically taking a strong cytochrome P450, family 3, subfamily A, polypeptide 4 (CYP3A) inhibitor or inducer and cannot be switched to an alternative agent at least 7 days prior to idelalisib or ibrutinib initiation that in the opinion of investigator/treating physicians precludes utilization of either Ibrutinib or Idelalisib; caution is recommended for patients taking moderate inhibitors of CYP3A
* CLL ARMS (ARM A and ARM C) FOR COMBINATION THERAPY INCLUDING IDELALISIB ARM:
* Is chronically taking a sensitive CYP3A substrate or a CYP3A substrate with a narrow therapeutic index and cannot be switched to an alternative agent at least 7 days prior to study initiation that in the opinion of investigator/treating physicians precludes utilization of idelalisib
* A history of chronic diarrhea, colitis, or intestinal perforation that in the opinion of the investigator precludes utilization of idelalisib