Veliparib, Pembrolizumab, and Combination Chemotherapy in Treating Patient With Locally Advanced Rectal Cancer

Inclusion Criteria:

* The patient must have signed and dated an Institutional Review Board (IRB)-approved consent form that conforms to federal and institutional guidelines
* Eastern Cooperative Oncology Group (ECOG) performance status of 0, 1, or 2
* Diagnosis of adenocarcinoma of the rectum with the major portion of the tumor intact; Note: prior to randomization, the investigator must specify and document each of the following:

  * Distance of the lowest tumor margin from the anal verge; and
  * Intent for sphincter sparing or non-sphincter sparing surgical resection according to the primary surgeon; and
  * The majority of the untreated tumor must be \< 12 cm from the anal verge or below the peritoneal reflection as determined by the treating surgeon
* The tumor must be clinically determined to be locally advanced stage II or stage III rectal cancer, and must also meet any ONE of the following criteria:

  * Distal location (as defined by measurement on magnetic resonance imaging \[MRI\], transrectal ultrasound \[ERUS\]/pelvic computed tomography \[CT\] \[with IV contrast\] scan or palpable on digital rectal examination \[DRE\]): cT3-4 =\< 5 cm from the anal verge, any N
  * Bulky: any cT4 or evidence that the tumor is adjacent to (defined as within 3 mm of) the mesorectal fascia on MRI or ERUS/pelvic CT (with IV contrast) scan
  * High risk for metastatic disease with 4 or more regional lymph nodes (cN2); clinical nodal or "cN" status for eligibility includes the total number of nodes (N2 = 4 or more) in the mesorectal and superior rectal stations measuring \>= 1.0 cm in any axis on cross sectional or endoscopic imaging; Note: nodes must measure 1.0 cm or greater to be considered positive for this eligibility requirement
  * Not a candidate for sphincter-sparing surgical resection prior to neoadjuvant therapy (as planned by the primary surgeon)

    * Note: clinical stage of the primary tumor and nodes may be determined locally by rectal endoscopic ultrasound or pelvic MRI (pelvic MRI is strongly preferred); CT scan with IV contrast is acceptable provided there is evidence of T4 and/or N2 disease
* Patients must have the ability to swallow and retain oral medication
* Absolute neutrophil count (ANC) must be \>= 1200/mm\^3 (within 28 days before randomization)
* Platelet count must be \>= 100,000/mm\^3 (within 28 days before randomization)
* Hemoglobin must be \>= 10 g/dL (within 28 days before randomization)
* Total bilirubin must be =\< ULN (upper limit of normal) for the lab unless the patient has a bilirubin elevation \> ULN to 1.5 x ULN due to Gilbert's disease or similar syndrome involving slow conjugation of bilirubin (within 28 days before randomization)
* Alkaline phosphatase must be =\< 3 x ULN for the lab (within 28 days before randomization)
* Aspartate aminotransferase (AST) must be =\< 3 x ULN for the lab (within 28 days before randomization);

  * Note: if alanine aminotransferase (ALT) is performed instead of AST (per institution's standard practice), the ALT value must be =\< 3 x ULN; if both were performed, the AST must also be =\< 3 x ULN; if AST and/or ALT is \>= ULN but =\< 3 x ULN, serologic testing for hepatitis B and C must be performed and results for viral infection must be negative
* Serum creatinine =\< ULN for the lab and measured or calculated creatinine clearance \> 60 mL/min (within 28 days before randomization)
* Serum potassium, magnesium, and calcium levels within 28 days before randomization must be within normal limits (WNL) for the lab
* International normalized ratio of prothrombin time (INR) within 28 days before randomization must be =\< ULN for the lab; patients who are therapeutically treated with an agent such as warfarin may participate if they are on a stable dose and no underlying abnormality in coagulation parameters exists per medical history
* Patients with acquired immunodeficiency syndrome (AIDS-related illnesses) or known human immunodeficiency virus (HIV) disease must:

  * Have a cluster of differentiation (CD)4 count \>= 200 cells/uL within 30 days before beginning study therapy
  * Be off all antiretroviral therapy (prophylaxis/treatment) more than 60 days before beginning study therapy, and
  * Have no evidence of opportunistic infections
* Pregnancy test (urine or serum beta-human chorionic gonadotropin \[HCG\]) done within 72 hours before randomization must be negative (for women of childbearing potential only); if urine pregnancy results are positive or cannot be confirmed as negative, a serum pregnancy test will be required

Exclusion Criteria:

* Rectal cancer histology other than adenocarcinoma (i.e., sarcoma, lymphoma, squamous cell carcinoma, mucosal melanoma, etc.)
* Definitive clinical or radiologic evidence of metastatic disease; required imaging studies must have been performed within 28 days prior to randomization; Note: Distant clinical staging to exclude patients with overt metastatic disease is determined by:

  * Chest: CT scan (preferred); chest x-ray posterioranterior (PA) and lateral (acceptable); or positron emission tomography (PET) scan (acceptable)
  * Abdomen: CT scan with IV contrast (preferred); or MRI (acceptable)
  * Pelvis: MRI (preferred) or CT scan with IV contrast (acceptable)

    * (It is recommended that the same imaging tests that are performed before randomization be used at follow-up time points; Note: CT scans of the abdomen and pelvis must be performed with IV contrast)
* History of prior invasive rectal malignancy, regardless of disease-free interval
* Cardiac disease that would preclude the use of any of the drugs included in the GI002 treatment regimen; this includes but is not limited to:

  * Clinically unstable cardiac disease, including unstable atrial fibrillation, symptomatic bradycardia, unstable congestive heart failure, active myocardial ischemia, or indwelling temporary pacemaker
  * Ventricular tachycardia or supraventricular tachycardia that requires treatment with class Ia antiarrhythmic drugs (e.g., quinidine, procainamide, disopyramide) or class III antiarrhythmic drug (e.g., sotalol, amiodarone, dofetilide); use of other antiarrhythmic drugs is permitted
  * Second- or third-degree atrioventricular (AV) block unless treated with a permanent pacemaker
  * Complete left bundle branch block (LBBB)
  * History of long QT syndrome
  * Corrected QT (QTc) \>= 450 ms
* Sensory or motor neuropathy \>= grade 2
* History of, or any evidence of active, non-infectious pneumonitis
* Active inflammatory bowel disease (i.e., patients requiring current medical interventions or who are symptomatic) or have a history of abdominal surgery or other medical condition that may, in the opinion of the treating physician, interfere with gastrointestinal motility or absorption
* Active autoimmune disease that has required systemic treatment within the past 2 years (i.e., with use of modifying agents, corticosteroids, or immunosuppressive drugs); replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment
* History of active TB (Bacillus tuberculosis)
* Active or chronic infection requiring systemic therapy
* Diagnosis of immunodeficiency or receiving systemic steroid therapy or any other form of immunosuppressive therapy within 7 days prior to the first dose of study therapy; the use of physiologic doses of corticosteroids may be approved after consultation with the study principal investigator (PI)
* Active seizure disorder uncontrolled by medication
* Any antineoplastic therapy for this cancer before randomization
* Synchronous colon cancer
* Other invasive cancer within 5 years before randomization; exceptions are colonic polyps, non-melanoma skin cancer or carcinoma-in-situ of the cervix
* Antineoplastic therapy (e.g. chemotherapy or targeted therapy) for other invasive cancer within 5 years before randomization; (for the purposes of this study, hormonal therapy is not considered chemotherapy)
* Prior treatment with an investigational compound being tested in this study (e.g., poly ADP ribose polymerase \[PARP\] inhibitor, anti-PD-1, anti-PD-L1, or anti-PD-L2)
* Receipt of live vaccination within 28 days before randomization; seasonal flu vaccines that do not contain live virus are permitted
* Major surgery within 4 weeks before randomization
* Any therapeutic pelvic radiation
* Known homozygous DPD (dihydro pyrimidine dehydrogenase) deficiency
* Any of the following because this study involves agents that have known or potential genotoxic or mutagenic, and teratogenic effects:

  * Pregnant women
  * Nursing women who are unwilling to discontinue nursing
  * Men or women of childbearing potential who are unwilling to employ adequate contraception (e.g., hormonal or barrier method of birth control; abstinence) for the duration of study treatment and for 4 months after the last dose of study therapy
* Co-morbid illnesses or other concurrent disease that, in the judgement of the clinician obtaining informed consent, would make the patient inappropriate for entry into this study or interfere significantly with the proper assessment of safety and toxicity of the prescribed regimens or prevent required follow-up