Pembrolizumab and Autologous Dendritic Cells for the Treatment of Refractory Colorectal Cancer (CRC)

Inclusion Criteria:

* Age \>= 18 years of age
* Recurrent and/or metastatic unresectable microsatellite stable (MSS) colorectal cancer
* At least 2 target lesions present per Response Evaluation Criteria in Solid Tumours (RECIST) 1.1 at least one of which is amenable to biopsy and injection
* Prior treatment with, contra-indication to, or refusal of a fluoropyrimidine, irinotecan, oxaliplatin and an anti-EGFR targeted therapy (if RAS wt), as well as avastin/bevacizumab
* PD-1/PD-L1/PD-L2 treatment naïve patients are eligible
* Have an Eastern Cooperative Oncology Group (ECOG) performance status of 0-1
* Platelet \>= 75,000/uL
* Hemoglobin \>= 8 g/dL (without transfusion in the past 14 days)
* Absolute neutrophil count (ANC) \>= 1500/uL
* Estimated creatinine clearance (Cockcroft Gault) \>= 30 mL/min/ for participant with creatinine levels \> 1.5 x institutional upper limit of normal (ULN)
* Total bilirubin: =\< 2 x ULN OR direct bilirubin =\< ULN for participants with total bilirubin levels \> 2 x ULN
* Aspartate aminotransferase (AST) serum glutamic-oxaloacetic transaminase (SGOT) and alanine aminotransferase (ALT) serum glutamic-pyruvic transaminase (SGPT) =\< 3 x institutional ULN (=\< 5 x ULN for participants with liver metastases)
* Women of childbearing potential must agree to use acceptable birth control methods for the duration of the study and until persistence of the study drug is no longer detected in the peripheral blood: This may be a period of several years. Methods for acceptable birth control include condoms, diaphragm or cervical cap with spermicide, intrauterine device, and hormonal contraception; it is recommended that a combination of two methods be used

  * NOTE: If the risk of conception exists, patients must continue to use highly effective contraception for at least two years following the last study treatment administration
* A male participant must agree to use a contraception during the treatment period and for at least 1 year after the last dose of study treatment and refrain from donating sperm during this period
* Participant must understand the investigational nature of this study and sign an Independent Ethics Committee/Institutional Review Board approved written informed consent form prior to receiving any study related procedure

Exclusion Criteria:

* Patients currently treated with systemic immunosuppressive agents. If a patient is currently on steroids, they must be on a steroid dose less than or equal to an equivalent prednisone dose of 10 mg daily
* Patients with active autoimmune disease, requiring ongoing immunosuppressive therapy or history of transplantation
* Has known active central nervous system (CNS) metastases and/or carcinomatous meningitis. Participants with previously treated brain metastases may participate provided they are radiologically stable, i.e. without evidence of progression for at least 4 weeks by repeat imaging (note that the repeat imaging should be performed during study screening), clinically stable and without requirement of steroid treatment for at least 14 days prior to first dose of study intervention
* Has received prior therapy with an anti-PD-1, anti-PD-L1, or anti-PD-L2 agent or with an agent directed to another stimulatory or co-inhibitory T-cell receptor
* Has received prior chemotherapy or radiotherapy within 2 weeks of start of study intervention. All chemotherapy or radiation-related toxicities must be resolved to =\< grade 1, not require corticosteroids, and not have had radiation pneumonitis. A 1-week washout is permitted for palliative radiation (=\< 2 weeks of radiotherapy) to non-CNS disease
* Is currently participating in or has participated in a study of an investigational agent or has used an investigational device within 4 weeks prior to the first dose of study intervention
* Has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of study drug
* Has severe hypersensitivity (\>= grade 3) to pembrolizumab and/or any of its excipients
* Has a history of (non-infectious) pneumonitis/interstitial lung disease that required steroids or has current pneumonitis/interstitial lung disease
* Has an active infection requiring systemic therapy
* Has a known history of human immunodeficiency virus (HIV) infection. Note: No HIV testing is required
* Has a known history of hepatitis B (defined as hepatitis B surface antigen \[HBsAg\] reactive) or known active hepatitis C virus (defined as hepatitis C virus \[HCV\] ribonucleic acid \[RNA\] \[qualitative\] is detected) infection. Note: no testing for hepatitis B and hepatitis C is required
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
* Pregnant or nursing female participants
* Unwilling or unable to follow protocol requirements
* Any condition which in the investigator's opinion deems the participant an unsuitable candidate to receive study drug