Venetoclax With Combination Chemotherapy in Treating Patients With Newly Diagnosed or Relapsed or Refractory Acute Myeloid Leukemia

Inclusion Criteria:

* Diagnosis of AML by World Health Organization (WHO) criteria. Patients with high risk myelodysplastic syndrome (MDS) as defined by the presence of \>= 10% blasts are also eligible at the discretion of the principal investigator
* Patients older than 65 who are deemed fit to receive intensive chemotherapy by the treating physician will be eligible after discussion with the principal investigator (PI).
* Eastern Cooperative Oncology Group (ECOG) performance status of =\< 2
* Creatinine clearance \>= 30 mL/min based on the Cockcroft-Gault equation
* Total bilirubin \< 1.5 x upper limit of normal (ULN) unless increase is due to Gilbert's disease or leukemic involvement
* Aspartate aminotransferase (AST) and/or alanine aminotransferase (ALT) \< 3 x ULN unless considered due to leukemic involvement
* Ability to understand and provide signed informed consent
* Male subjects must agree to refrain from unprotected sex and sperm donation from initial study drug administration until 90 days after the last dose of study drug
* Only patients who are relapsed, refractory, or intolerant of standard AML therapy will be eligible for Part 1 (minimum of 1 prior line of AML-directed therapy)

Exclusion Criteria:

* Patients with t(15;17) karyotypic abnormality or acute promyelocytic leukemia (French-American-British \[FAB\] class M3-AML)
* Patients having received any prior BCL2 inhibitor therapy
* Subject has known active central nervous system (CNS) involvement with AML
* Patients with New York Heart Association (NYHA) class III or IV congestive heart failure or left ventricular ejection fraction (LVEF) \< 40% by echocardiogram or multi-gated acquisition (MUGA) scan
* Patients with a history of myocardial infarction within the last 6 months or unstable / uncontrolled angina pectoris or history of severe and/or uncontrolled ventricular arrhythmias
* Patients with known infection with human immunodeficiency virus (HIV) or active hepatitis B or C
* Patients with known dysphagia, short-gut syndrome, or other conditions that would affect the ingestion or gastrointestinal absorption of drugs administered orally
* Subject has any other significant medical or psychiatric history that in the opinion of the investigator would adversely affect participation in this study
* Subject has a white blood cell count \> 25 x 10{9}/L. (Note: hydroxyurea is permitted to meet this criterion)
* Nursing women, women of childbearing potential (WOCBP) with positive urine pregnancy test, or women of childbearing potential who are not willing to maintain adequate contraception (a) appropriate method(s) of contraception include oral or injectable hormonal birth control, intrauterine device (IUD), and double barrier methods (for example a condom in combination with a spermicide)