Trifluridine/Tipiracil and Talazoparib for the Treatment of Patients With Locally Advanced or Metastatic Colorectal or Gastroesophageal Cancer

Inclusion Criteria:

* Histologically or cytologically confirmed CRC or EGC adenocarcinoma that is locally advanced or metastatic (Cohort A); histologically or cytologically confirmed p53mt/RASonc (Cohort B1) or p53mt/RASwt CRC (Cohort B2) that is locally advanced or metastatic. Patients with adenocarcinoma histology only are allowed to participate.
* Has received at least one prior line of therapy with progression or intolerance
* Have measurable disease per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria present
* Have an Eastern Cooperative Oncology Group (ECOG) performance status of 0-1
* Life expectancy \>= 3 months by investigator assessment
* Hemoglobin \>= 9 g/dL
* Absolute neutrophil count \>= 1500/mm\^3
* Platelet count \>= 100,000/mm\^3 without transfusion or growth factor support
* Creatinine \< 1.5 upper limit of normal (ULN) or creatinine clearance \> 60 mL/min
* Total bilirubin \< 1.5 x ULN
* Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 2.5 x ULN or \< x 5 ULN in the presence of liver metastasis
* Albumin \> 3 g/dL
* Ability to swallow oral medications
* Participants of child-bearing potential must agree to use adequate contraceptive methods (e.g., hormonal or barrier method of birth control; abstinence) prior to study entry. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately
* Participant must understand the investigational nature of this study and sign an Independent Ethics Committee/Institutional Review Board approved written informed consent form prior to receiving any study related procedure

Exclusion Criteria:

* Systemic antineoplastic therapy within 2 weeks prior to day -14 (Dose Escalation, Cohort A) or Cycle 1 day 1 (Dose Expansion, Cohorts B1 and B2) or within the past 6 weeks if this treatment is mitomycin C or nitrosourea
* Radiotherapy within the past 2 weeks excluding palliative radiotherapy to painful bone lesions
* Prior treatment with PARP inhibitor, FUDR or FTD/TPI
* Any condition that in the investigator's opinion can limit absorption of FTD/TPI or talazoparib from the gastrointestinal (GI) tract
* Gastrointestinal obstruction (without diversion) or perforation within 4 weeks from initiation of day -14 (Dose Escalation, Cohort A) or Cycle 1 Day 1 (Dose Expansion, Cohorts B1 and B2.
* Refractory ascites (requiring weekly or more frequent paracentesis or permanent indwelling peritoneal catheter)
* Untreated central nervous system (CNS) disease. Patients with leptomeningeal disease are ineligible but patients with treated, stable CNS metastasis for at least 4 weeks are allowed to participate
* Significant cardiac disease defined as congestive heart failure stage III or IV (New York Heart Association \[NYHA\]), acute coronary event, cerebrovascular event, peripheral arterial embolic event, venous thromboembolic event (pulmonary embolism or lower extremity deep vein thrombosis), or ventricular arrhythmia within the past 3 months
* Other malignancy requiring active therapy
* Presence of toxicities from prior therapy of grade 2 or higher
* Active infection requiring antibiotic therapy
* Known human immunodeficiency virus (HIV) or hepatitis B infection or untreated hepatitis C infection. Patients with treated hepatitis C infection and undetectable viral load are allowed to participate
* Any history of myelodysplastic syndrome, acute leukemia, or bone marrow transplant
* Pregnant or nursing female participants
* Unwilling or unable to follow protocol requirements
* Any condition which in the investigator's opinion deems the participant an unsuitable candidate to receive study drug