A Phase 1 Study of Alisertib Participants With Advanced Solid Tumors Including Castration-Resistant Prostate Cancer Receiving a Standard Docetaxel Regimen

Inclusion Criteria:

Each participant must meet all of the following inclusion criteria to be enrolled in the study:

* 18 years or older
* Histologically or cytologically confirmed advanced tumors and candidates for docetaxel treatment
* Measurable or evaluable disease is required. Participants must have clinical evidence of progressive disease or persistent disease
* Participants with castration-resistant prostate cancer (CRPC) are required to have

  * Pathologically confirmed adenocarcinoma of the prostate
  * Evidence of metastatic disease on bone scan or other imaging. Participants with prostate-specific antigen (PSA) elevation as the only manifestation of disease are not eligible.
  * Progressive disease after at least 1 hormonal treatment with documented testosterone levels less than 50 ng/dl
  * Concurrent use of an agent for testosterone suppression (e.g., luteinizing hormone-releasing hormone \[LHRH\] agonist) is required if the participants has not been surgically castrated
* Eastern Cooperative Oncology Group (ECOG) performance status 0-2
* Recovered to less than or equal to Grade 1 toxicity (CTCAE), to participant's baseline status (except alopecia) or deemed irreversible from the effects of prior cancer therapy and must have evidence of progressive or persistent disease
* Adequate bone marrow, liver and renal function
* Any use of opiates must be stable for at least 2 weeks prior to study entry
* Female participants who are postmenopausal for at least 1 year OR are surgically sterile OR if of childbearing potential, agree to practice 2 effective methods of contraception at the same time
* Male participants who agree to practice effective barrier contraception during the entire study and through 6 months after the last dose of study drug OR agree to abstain from heterosexual intercourse
* Voluntary written consent
* Willing to comply with scheduled visits, treatment plan, laboratory tests and other trial procedures
* Suitable venous access for blood sampling

Exclusion Criteria:

Participants meeting any of the following exclusion criteria are not to be enrolled in the study:

* Female participants who are lactating or pregnant
* Antineoplastic therapy or any experimental therapy within 21 days before the first dose of alisertib
* Prior or current investigational therapies within 4 weeks before the first dose of MLN8237
* Concurrent investigational treatment of treatment with any investigational products within 28 days before the first dose of alisertib
* Radiotherapy to greater than 40% of bone marrow or any radiotherapy (except localized, small field radiation) within 4 weeks prior to enrollment, unless reviewed and approved by the medical monitor
* Nitrosoureas or mitomycin-C within 6 weeks before the first dose of alisertib.
* Autologous stem cell transplant within 3 months before the first dose of alisetib, or prior allogeneic stem cell transplant at any time.
* Use of enzyme-inducing antiepileptic drugs such as phenytoin, carbamazepine or phenobarbital, or rifampin, rifabutin, rifapentine or St. John's wort within 14 days prior to the first dose of alisertib
* For CRPC participants:

  * Radiotherapy or antiandrogen therapy for prostate cancer within 4 weeks prior to enrollment
  * Prior treatment with antineoplastic chemotherapy or radioisotopes for advanced prostate cancer
  * Use of products known to affect PSA levels within 4 weeks of enrollment
* Major surgery within 4 weeks of study enrollment
* Uncontrolled high blood pressure
* Participants with abnormal gastric or bowel function or who require continuous treatment with antacids or proton pump inhibitors
* Participants receiving chronic steroid therapy other than the following: low dose steroid for the control of nausea and vomiting, topical steroid, inhaled steroid or use of dexamethasone
* Known severe hypersensitivity to docetaxel or other drugs formulated in polysorbate 80
* Comorbid condition or unresolved toxicity that would preclude administration of docetaxel
* Prior history of Grade 2 or greater neurotoxicity or any toxicity that has not resolved to Grade 1 or below
* Symptomatic brain or other CNS metastasis
* Diagnosis or treatment of another malignancy within 2 years preceding first dose of study drug except nonmelanoma skin cancer or in situ malignancy completely resected
* Known history of human immunodeficiency virus (HIV) infection, hepatitis B, or hepatitis C
* Participants requiring full systemic anticoagulation
* Prior allogeneic bone marrow or other organ transplant
* Active infection requiring systemic therapy within 14 days preceding first dose, or other serious infection
* History of hemorrhagic or thrombotic cerebrovascular event in the past 12 months
* Serious medical or psychiatric illness that could interfere with protocol completion
* Inability to swallow oral medication
* Prior treatment with more than 3 myelosuppressive cytotoxic chemotherapy regimens
* Prior treatment with more than 1 prior taxane-containing regimen