Total Marrow and Lymphoid Irradiation, Fludarabine, and Melphalan Before Donor Stem Cell Transplant in Treating Participants With High-Risk Acute Leukemia or Myelodysplastic Syndrome

Inclusion Criteria:

* Eligible patients with a histopathological confirmed diagnosis of hematologic malignancy in one of the following categories:

  * Acute myelogenous leukemia:

    * Patients with de novo or secondary disease in unfavorable risk group including poor risk cytogenetics according to National Comprehensive Cancer Network (NCCN) guidelines for AML i.e., monosomal karyotype, -5, 5q-, -7, 7q-, 11q23-non t (9;11), inv (3), t (3;3),t (6;9), t (9;22) and complex karyotypes (≥ 3 unrelated abnormalities), or all patient in intermediate risk groups accept patients with FLT3-NPM1+ disease
    * Patients with active disease
    * Patients with chemosensitive active disease
  * Acute lymphocytic leukemia:

    * Patients with de novo or secondary disease according to NCCN guidelines for ALL hypoploidy (\< 44 chromosomes); t (v;11q23): MLL rearranged; t (9;22) (q34;q11.2); complex cytogenetics (5 or more chromosomal abnormalities); high white blood cell (WBC) at diagnosis (≥ 30,000 for B lineage or ≥ 50,000 for T lineage); iAMP21loss of 13q, and abnormal 17p
    * Patients with active disease
    * Patients with chemosensitive active disease
  * Myelodysplastic syndrome in high-intermediate (int-2) and high risk categories
* Patients ≥ 12 years and \< 55 years are also included if they are not candidates for myeloablative conditioning regimens due to comorbidities
* Karnofsky or Lansky performance status of ≥ 70
* A pretreatment measured creatinine clearance (absolute value) of ≥ 60 ml/minute
* Patients must have a serum bilirubin ≤ 2.0 mg/dl
* Serum glutamic-oxaloacetic transaminase (SGOT) and serum glutamic-pyruvic transaminase (SGPT) ≤ 2.5 times the institutional upper limits of normal
* Ejection fraction measured by echocardiogram or multigated acquisition (MUGA) ≥ 50%
* Carbon monoxide diffusing capacity (DLCO) and forced expiratory volume FEV1 \> 50% predicted
* PATIENT-SPECIFIC INCLUSION CRITERIA
* Patients should have discontinued all previous intensive therapy, chemotherapy or radiotherapy for 2 weeks prior to commencing therapy on this study

  * NOTE: low dose chemotherapy or maintenance chemotherapy given within 7 days of planned study enrollment is permitted; these include hydroxyurea, 6-meraptopurine, oral methotrexate, vincristine, oral etoposide, and tyrosine kinase inhibitors (TKIs); FLT-3 inhibitors can also be given up to 3 days before conditioning regimen
  * All patients with prior radiation treatment to the lung, liver, and kidney will be excluded; for other scenarios of prior radiation treatment, up to 2000 cGY at 2 gray Gy per day will be allowed; inclusion of patients with previous radiation exposure will be determined based on the radiation oncologist MD evaluation and judgment
* DONOR: Arm A: All candidates for this study must have an human leukocyte antigen (HLA) (A, B, C, and DR) identical sibling who is willing to donate primed blood stem cells (preferred) or bone marrow, or have a 10/10 (A, B, C, DR and DQ) allele matched unrelated donor; DQ or DP mismatch is allowed per discretion of the principal investigator
* DONOR: Arm B: The recipient must have a related donor genotypically HLA-A, B, C and DRB1 loci haploidentical to the recipient; no HLA matched sibling or matched unrelated donor is available; DSA is allowed with desensitization done if recommended by donor selection committee (DSC) per City of Hope (COH) standard operating procedures (SOP)
* DONOR: Both arms: All donors in both arms should be evaluated and approved by DSC

Exclusion Criteria:

* Having any uncontrolled illness including ongoing or active bacterial, viral or fungal infection
* Receiving any investigational agents or concurrent biological, intensive chemotherapy or radiation therapy for the previous 2 weeks from conditioning

  * NOTE: low dose chemotherapy or maintenance chemotherapy given within 7 days of planned study enrollment is permitted; these include: hydroxyurea, 6-meraptopurine, oral methotrexate, vincristine, oral etoposide, and tyrosine kinase inhibitors (TKIs), FLT-3 inhibitors can also be given up to 3 days before conditioning regimen
* History of allergic reactions attributed to compounds of similar chemical or biologic composition to any in the regimen
* Patients with other malignancies are ineligible for this study, other than non-melanoma skin cancers
* The recipient has another medical problem or neurologic/psychiatric dysfunction which would impair his/her ability to be compliant with the medical regimen and to tolerate transplantation or would prolong hematologic recovery in which the opinion of the principal investigator would place the recipient at unacceptable risk
* Patients may not have had a prior autologous or allogeneic transplant
* Patients may not have received more than 3 prior lines of intensive chemotherapy, where the regimen intent was to induce remission
* In the opinion of the principal investigator (PI), the participant has a condition that will preclude them from complying with study treatment
* Women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control or abstinence) prior to study entry and for six months following duration of study participation; should a woman become pregnant or suspect that she is pregnant while participating on the trial, she should inform her treating physician immediately
* All subjects must have the ability to understand and the willingness to sign a written informed consent; they are to give voluntary written informed consent before performance if any study-related procedure not part of normal medical care, with the understanding that consent may be withdrawn by the subject at any time without prejudice to future medical care; cognitively impaired subjects will be included only if their guardian or legal representative agrees to sign the written informed consent
* DONOR: Donor selection for both arms must be approved by the donor selection committee
* DONOR: Evidence of active infection
* DONOR: Medical or physical reason which makes the donor unlikely to tolerate or cooperate with growth factor therapy or leukapheresis
* DONOR: Factors which place the donor at increased risk for complications from leukapheresis or granulocyte-colony stimulating factor (G-CSF) therapy could be harvested for bone marrow (BM) if safer for the donor and if approved by the principal investigator (PI)
* DONOR: Human immunodeficiency virus (HIV) positive