Personalized Peptide Vaccine in Treating Patients With Advanced Pancreatic Cancer or Colorectal Cancer

Inclusion Criteria:

* Metastatic CRC or PDA planned to or have undergone complete surgical resection (metastatectomy) and also for PDA participants with localized disease planned for primary tumor resection.
* Any lines (including zero) of therapy prior to tissue harvest.
* Adequate tumor tissue availability
* Adults (age ≥ 18)
* ECOG PS 0-1
* Life expectancy \>12 months for Cohort C and \>9 months for Cohort D
* Adequate organ and marrow function
* Ability to understand and the willingness to sign a written informed consent document.
* As the effects of a peptide based vaccine, pembrolizumab or APX005M on the developing human fetus are unknown, women of child-bearing potential and men must agree to use adequate contraception at study entry and for the duration of study participation. Should a woman become pregnant or suspect she is pregnant, she should inform her treating physician immediately. Unless surgically sterile by bilateral tubal ligation or vasectomy of partner(s), sexually active participants must use birth control during and for \>120 days after the study. Abstinence is also an acceptable form of birth control.
* For Cohort C only: participants must have metastatic CRC and are planned to or have undergone complete metastecomy/ies and agree to have post-operative blood draw for ctDNA testing within 6 weeks following surgical resection.

Exclusion Criteria:

* History of HIV or AIDS
* Patients with brain metastasis
* Serious autoimmune conditions
* Use of chronic immune suppressive medications.
* Uncontrolled concurrent illness including, but not limited to ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements.
* Women of child bearing potential who are pregnant or breastfeeding. Women with a positive pregnancy test at enrollment or prior to administration of vaccine. .
* Has history of (non-infectious) pneumonitis that required steroids, evidence of interstitial lung disease or active, non-infectious pneumonitis.
* Known history of active TB (Bacillus Tuberculosis).
* Hypersensitivity to therapy drugs or their components.
* Has a known additional malignancy that is progressing or requires active treatment.
* Active coagulopathy.
* History of arterial thrombosis within 3 months prior to starting study treatment.
* History of New York Heart Association Class 3-4 heart failure or myocardial infarction within 6 months prior to starting therapy.
* Known history of Hepatitis B or known active Hepatitis C.