Enzalutamide and Decitabine in Treating Patients With Metastatic Castration Resistant Prostate Cancer

Inclusion Criteria:

* Histological or cytological documentation of diagnosis of prostate cancer, all histological sub-types included.
* Documented progressive metastatic castrate resistant prostate cancer (mCRPC) based on at least one of the following criteria:

  * PSA progression defined as 25% increase over baseline value with an increase in the absolute value of at least 2 ng/mL that is confirmed by another PSA level with a minimum of a 1week interval and a minimum PSA of 2 ng/mL
  * Soft-tissue progression defined as an increase \>= 20% in the sum of the longest diameter (LD) of all target lesions based on the smallest sum LD since treatment started or the appearance of one or more new lesions
  * Progression of bone disease (evaluable disease) or (new bone lesion\[s\]) by bone scan
  * If on an anti-androgen, must have documented progression 6 weeks after stopping anti-androgen therapy
* Willing to undergo a biopsy, if readily available biopsy site present, i.e., nodal or visceral metastasis (if adequate formalin-fixed, paraffin-embedded (FFPE) archival mCRPC samples are not available (or biopsy was taken longer than 6 months from start of study treatment), a fresh pre-treatment mCRPC biopsy needs to be obtained)
* Have an Eastern Cooperative Oncology Group (ECOG) performance status of 0-1
* Have testosterone \< 50 ng/dL. Note: Patients must continue primary androgen deprivation with an luteinizing hormone-releasing hormone (LHRH) analogue (agonist or antagonist) if they have not undergone orchiectomy
* White blood cells \>= 1.5 x 10\^9/L (obtained within 14 days prior to treatment start)
* Platelets (UNVPLT) \>= 100 x 10\^9/L (obtained within 14 days prior to treatment start)
* Hemoglobin (HGB) \>= 9 g/dL (obtained within 14 days prior to treatment start)
* Potassium (K), total calcium (CA) (corrected for serum albumin), magnesium, sodium (NA) and phosphorus within normal limits for the institution or corrected to within normal limits with supplements before first dose of study medication (obtained within 14 days prior to treatment start)
* International normalized ratio (INR) =\< 1.5 (obtained within 14 days prior to treatment start)
* Serum creatinine (CREAT) =\< 1.5 mg/dL or creatinine clearance \> 50 mL/min (obtained within 14 days prior to treatment start)
* Alanine aminotransferase (ALT) and aspartate aminotransferase (AST) =\< 2.5 x ULN. If the patient has liver metastases, ALT and AST must still be =\< 2.5 x ULN. Patients with liver metastases and AST/ALT above this limit will not be enrolled (obtained within 14 days prior to treatment start)
* Total serum bilirubin =\< 1.5 x ULN; or total bilirubin (TBILI) =\< 3.0 x ULN with direct bilirubin within normal range in patients with well documented Gilbert?s syndrome (obtained within 14 days prior to treatment start)
* Ability to swallow and retain oral medication (without crushing, dissolving or chewing tablets)
* Phase Ib only: Prior enzalutamide treatment and/or other approved treatments for CRPC are acceptable
* Phase II only: Participants MUST be treatment na?ve in the CRPC setting: i.e., no prior exposure to abiraterone acetate other specific CYP-17 inhibitors; no prior exposure to enzalutamide or investigational androgen receptor (AR) targeted agents; and no prior exposure to chemotherapy and or RAD-223
* Sexually active males must agree to use a condom during intercourse while taking the study drug and for at least 3 months after stopping study treatment. Sexually active males should not father a child during this period. A condom is required to be used by vasectomized men in order to prevent delivery of the drug via seminal fluid. Should a woman become pregnant or suspect she is pregnant while her partner is participating in this study, she should inform her treating physician immediately
* Participant or legal representative must understand the investigational nature of this study and sign an Independent Ethics Committee/Institutional Review Board approved written informed consent form prior to receiving any study related procedure

Exclusion Criteria:

* Phase II only: Prior exposure to abiraterone acetate
* Phase II only: Prior exposure to hypomethylating agents like azacytidine or decitabine
* Phase II only: Prior chemotherapy for castration resistant disease. Chemotherapy given in the castration-sensitive setting is permissible if stopped at least 4 weeks prior to treatment start
* Phase II only: Prior isotope therapy with strontium-89, samarium or radium-223 within 12 weeks of treatment start
* Participants with known symptomatic brain metastases
* Participant has a concurrent malignancy or malignancy within 3 years of treatment start, with the exception of adequately treated, basal or squamous cell carcinoma ornon-melanomatous skin cancer
* Participant has a known history of human immunodeficiency virus (HIV) infection (testing not mandatory)
* Participant has clinically significant, uncontrolled heart disease and/or recent events including any of the following:

  * History of acute coronary syndromes (including myocardial infarction, unstable angina, coronary artery bypass grafting, coronary angioplasty, or stenting) or symptomatic pericarditis within 12 months prior to treatment start
  * History of documented congestive heart failure (New York Heart Association functional classification III-IV)
  * On screening 12 lead electrocardiography (ECG), any of the following cardiac parameters: bradycardia (heart rate \< 50 at rest), tachycardia (heart rate \> 90 at rest), PR interval \> 220 msec, QRS interval \> 109 msec or, Fridericia's correction formula (QTcF) \> 450 msec. Congenital long QT syndrome or family history of long QT syndrome
* Patient is currently receiving any of the following medications and cannot be discontinued 7 days prior to treatment start:

  * Known strong inducers or inhibitors of CYP3A4/5, including grapefruit, grapefruit hybrids and pummelos, star-fruit and, Seville oranges
  * Medications that have a known risk to prolong the QT interval or induce Torsades de Pointes
  * Herbal preparations/medications, dietary supplements
* Patient who has received radiotherapy =\< 4 weeks prior to start of treatment or limited field radiation for palliation =\< 2 weeks prior to treatment start and, who has not recovered to grade 1 or better from related side effects of such therapy (exceptions include alopecia) and/or in whom \>= 30% of the bone marrow was irradiated
* Patients with central nervous system (CNS) involvement
* Patients with seizure disorder
* Patient has not recovered from all toxicities related to prior anticancer therapies to National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) version 4.03 grade \< 1 (exception to this criterion: patients with any grade of alopecia are allowed to enter the study)
* Participant has any other concurrent severe and/or uncontrolled medical condition that would cause, in the investigator?s judgment, an unacceptable safety risk
* Unwilling or unable to follow protocol requirements
* Any condition which in the investigator?s opinion deems the participant an unsuitable candidate to receive study drug