IDO and PD-L1 Peptide Based Immune-Modulatory Therapeutic (IO102-IO103) in Combination With Pembrolizumab for BCG-Unresponsive or Intolerant, Non-Muscle Invasive Bladder Cancer

Inclusion Criteria:

* Adults \>= 18 years of age
* Histologically confirmed high-risk NMIBC (T1, high-grade Ta, or carcinoma in situ \[CIS\]/Tis). Mixed histologies are allowed if predominantly transitional cell histology. Archival tissue or planned cystoscopy within 28 day of planned initiation of treatment
* Maximally resected tumor on study entry
* Cystectomy ineligible or declined
* Two induction courses of BCG attempted, regardless of exact doses received
* ECOG (Eastern Cooperative Oncology Group) performance status score of 0 - 2
* Life expectancy \>= 6 months
* Absolute neutrophil count (ANC) \> 1000 cells/uL (=\< 14 days of the first study treatment)
* Platelet count \> 50,000/uL (=\< 14 days of the first study treatment)
* Hemoglobin \> 8 g/dL (=\< 14 days of the first study treatment)
* Aspartate aminotransferase (AST)/serum glutamic-oxaloacetic transaminase (SGOT), alanine aminotransferase (ALT)/serum glutamate pyruvate transaminase (SGPT) =\< 5 x upper limit of normal (ULN) (=\< 14 days of the first study treatment)
* Alkaline phosphatase =\< 5 x upper limit of normal (ULN) (=\< 14 days of the first study treatment)
* Total bilirubin =\< 2 x ULN (=\< 14 days of the first study treatment)
* Creatinine clearance \> 30 mL/min as measured using Cockcroft-Gault equation or the estimated glomerular filtration rate from the Modification of Diet in Renal Disease Study (=\< 14 days of the first study treatment)
* International normalized ratio (INR) or activated partial thromboplastin time (aPTT) =\< 1.5 X ULN unless the subject is receiving anticoagulant therapy. Individuals on anticoagulant therapy should have a prothrombin time (PT) or partial thromboplastin time (PTT) within therapeutic range of intended use and no history of severe hemorrhage
* Ability to understand and willingness to sign an informed consent document
* Ability to adhere to the study visit schedule and other protocol requirements
* For female patients of childbearing potential and male patients with partners of childbearing potential, agreement (by patient and/or partner) to use methods of contraception

Exclusion Criteria:

* Patients with a prior or concurrent malignancy whose natural history or treatment may, in the opinion of the investigator, have the potential to interfere with the safety or efficacy assessment of the investigational regimen
* Uncontrolled concomitant disease that in the opinion of the investigator would interfere with the patient's safety or compliance on trial
* Known history of positive test for human immunodeficiency virus (HIV) with CD4 \< 200 or acquired immunodeficiency syndrome (AIDS)-defining condition
* Known active tuberculosis
* Active infection requiring systemic therapy, including active or intractable urinary tract infection (UTI)
* Previous treatment with checkpoint inhibitors targeting either PD-(L)1 or CTLA-4
* Prior exposure to IO102 or IO103
* Received systemic chemotherapy, targeted small molecule therapy, or radiotherapy =\< 2 weeks before study treatment initiation
* Any adverse events from prior cancer therapy have resolved to grade =\< 1 according to Common Terminology Criteria for Adverse Events (CTCAE) version 5
* Congestive heart failure (as defined by New York Heart Association Functional Classification III or IV), unstable angina, serious uncontrolled cardiac arrhythmia, a myocardial infarction within 6 months prior to study entry or a history of myocarditis
* Any medical condition requiring systemic steroid equivalent to prednisone \> 10 mg daily or immunosuppressive therapy within 14 days or 5 half-lives prior to first dose of trial therapy. Patients with a history of autoimmune-related hypothyroidism on a stable dose of thyroid replacement hormone are eligible. Patients who have adrenal insufficiency and hypophysitis from prior immunotherapy if they are on stable medical replacement doses are eligible
* Received a live or live-attenuated vaccine =\< 30 days before the first dose of study treatment. Administration of killed vaccines, messenger ribonucleic acid (mRNA) based vaccines (e.g., COVID-19), and vector based vaccines are allowed
* Pregnant and/or breast feeding women. If a urine pregnancy test is positive or cannot be confirmed as negative, a serum pregnancy test will be required =\< 24 hours prior to planned treatment initiation
* Evidence of active interstitial lung disease or history of non-infectious pneumonitis requiring systemic steroids
* Known allergy or reaction to any component of either study drug formulation
* Any condition that would prohibit the understanding or rendering of informed consent
* Any condition that in the opinion of the investigator would interfere with the patient's safety or compliance while on trial