Bicalutamide and RO4929097 in Treating Patients With Previously Treated Prostate Cancer

Inclusion Criteria:

* Histologically confirmed prostate cancer

  * Stage D0 OR D1 disease (i.e., tumor originally diagnosed as being limited to the prostate and regional lymph nodes)
  * Has a rising PSA value after definitive local therapy (i.e., prostatectomy or radiotherapy) and no radiographic evidence of disease
* PSA progression after local treatment:

  * PSA values for patients after surgery must be ≥ 0.2 ng/mL, determined by two measurements, ≥ 1 month apart and ≥ 6 months after prostatectomy
  * PSA values for patients after radiotherapy must be ≥ 2.0 ng/mL above the nadir PSA achieved after radiotherapy, determined by two measurements at 1 month apart and ≥ 6 months after completion of the radiotherapy treatment (patients who received adjuvant or salvage radiotherapy after prostatectomy must have PSA of ≥ 0.2 ng/mL)
  * The first two PSA values, along with a third (study baseline) value must all be rising (i.e., there must be an overall rising trajectory, such that the third value cannot be lower than the first value)
* No metastatic disease on baseline bone scan and CT scan of the abdomen/pelvis
* ECOG performance status 0-2
* Life expectancy ≥ 6 months
* WBC ≥ 3,000/mm\^3
* ANC ≥ 1,500/mm\^3
* Platelet count ≥ 100,000/mm\^3
* Hemoglobin ≥ 9 g/dL
* Serum creatinine ≤ 1.5 times upper limit of normal (ULN) OR creatinine clearance ≥ 40 mL/min for patients with creatinine levels above normal
* Bilirubin normal
* AST and/or ALT ≤ 2.5 times ULN
* Serum total testosterone level ≥ 150 ng/dL
* No uncontrolled electrolyte abnormalities including hypocalcemia, hypomagnesemia, hyponatremia, hypophosphatemia, or hypokalemia, defined as less than the lower limit of normal despite adequate electrolyte supplementation
* Fertile patients must use two effective forms of contraception (i.e., barrier contraception and one other method of contraception) for 1 week before, during, and for ≥ 12 months after completion of study treatment
* Able to swallow tablets
* No malabsorption syndrome or other condition that would interfere with intestinal absorption
* No uncontrolled concurrent illness including, but not limited to, any of the following:

  * Ongoing or active infection
  * Symptomatic congestive heart failure
  * Unstable angina pectoris
  * A history of torsades de pointes
  * Cardiac arrhythmia other than chronic, stable atrial fibrillation
  * Psychiatric illness or social situation that would limit compliance with study requirements
* Baseline QTc ≤ 450 msec
* No serologic positivity for acute hepatitis A, acute or chronic hepatitis B, or acute or chronic hepatitis C
* No history of liver disease or other forms of hepatitis or cirrhosis
* No HIV-positive patients on combination antiretroviral therapy
* No serious concurrent systemic disorder that would compromise the safety of the patient or compromise the patient's ability to complete the study, at the discretion of the investigator
* No history of allergic reactions attributed to compounds of similar chemical or biologic composition to gamma-secretase inhibitor RO4929097 or bicalutamide
* Patients may not donate sperm or blood during or for ≥ 12 months after completion of study treatment
* No concurrent medications or food that may interfere with the metabolism of RO4929097 including grapefruit and fresh-squeezed grapefruit juice
* Recovered from adverse events to \< CTCAE grade 2
* At least 3 months since prior and no concurrent androgen-deprivation therapy (ADT)

  * Hormone-ablative treatment is only allowed in the neoadjuvant setting or in the setting of primary or salvage radiotherapy
  * No more than 36 months of neoadjuvant/ adjuvant ADT
* More than 4 weeks since prior chemotherapy or radiotherapy (6 weeks for nitrosoureas or mitomycin C)
* More than 4 weeks since prior and no concurrent estrogens, estrogen-like substances (i.e., PC-SPES, saw palmetto, or other herbal product that may contain phytoestrogens), or any other hormonal therapy (including flutamide, nilutamide, finasteride, ketoconazole, systemic corticosteroids, megestrol acetate, or cyproterone acetate)
* No other concurrent investigational agents
* No concurrent medications with narrow therapeutic indices that are metabolized by cytochrome P450 (CYP450), including warfarin sodium (Coumadin®)
* No concurrent medications that are strong inducers/inhibitors or substrates of CYP3A4
* No other investigational or commercial agents or therapies administered with the intent to treat the patient's malignancy, including chemotherapy, immunotherapy, hormonal cancer therapy, radiotherapy, or surgery for cancer
* No concurrent hormonal therapy with a leuteinizing hormone-releasing hormone (LHRH) agonist therapy (e.g., leuprolide or goserelin) or LHRH antagonist (e.g., abarelix)
* No concurrent growth factors (e.g., G-CSF), packed RBC transfusions, or platelet transfusions
* No concurrent antiarrhythmics or other medications known to prolong QTc