RO4929097 in Treating Patients With Recurrent and/or Metastatic Epithelial Ovarian Cancer, Fallopian Tube Cancer, or Primary Peritoneal Cancer

Inclusion Criteria:

* Patients must have histologically or cytologically confirmed ovarian epithelial carcinoma, fallopian tube carcinoma, or primary peritoneal carcinoma that is recurrent or metastatic; patients must have platinum resistant disease, as defined as treatment free interval less than 6 months post completion of platinum-based chemotherapy
* Patients must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded for non-nodal lesions and short axis for nodal lesions) as \>= 20 mm with conventional techniques or as \>= 10 mm with spiral CT scan

  * OR patients must have evidence of progression based on an elevated CA-125 (defined as a value of \> 2 x ULN documented on two separate determinations made \> 2 weeks apart, with the most recent being completed within 7 days prior to start of study treatment) if the physical exam is normal and maximum lesion diameter on the CT scan is \< 20 mm with conventional techniques or as \< 10 mm with spiral CT scan
* Patients must have completed any prior chemotherapy, radiotherapy or surgery \>= 4 weeks (6 weeks for nitrosoureas or mitomycin C) before study entry

  * Prior radiotherapy is allowed provided that there is documented progression (either locally or systemically)
  * Patients may have had up to 2 prior lines of chemotherapy for recurrent or metastatic disease
  * Toxic effects from prior therapy must have resolved to grade 1 or less except for peripheral neuropathy and alopecia
* Life expectancy of greater than 12 weeks
* ECOG performance status =\< 2 (Karnofsky \>= 60%)
* Patients must have normal organ and marrow function as defined below (within 7 days prior to start of study treatment):
* Hemoglobin \>= 90 g/L
* Leukocytes \>= 3.0 x 10\^9/L
* Absolute neutrophil count \>= 1.5 x 10\^9/L
* Platelets \>= 100 x 10\^9/L
* Total bilirubin =\< 2.5 ULN (institutional upper limit of normal)
* AST (SGOT)/ALT (SGPT) =\< 1.5 x institutional upper limit of normal (=\< 5 times the ULN for patients with liver metastases)
* Creatinine within normal institutional limits OR creatinine clearance \>= 60 mL/min/1.73 m\^2 for patients with creatinine levels above institutional normal (using Cockcroft-Gault Formula)
* All radiology studies must be performed =\< 4 weeks prior to the start of therapy
* No serious medical conditions such as myocardial infarction within 6 months prior to entry, congestive heart failure, unstable ventricular arrhythmia, uncontrolled hypertension, uncontrolled diabetes mellitus, uncontrolled psychotic disorders, serious infections, active peptic ulcer disease, psychiatric illness, or any other medical conditions that might be aggravated by treatment or limit compliance
* Currently no active, second malignancy other than non-melanoma skin cancers

  * NOTE: Patients are not considered to have a "current active" malignancy if they have completed anti-cancer therapy and are considered by their physician to be at less than 30% risk of relapse
* Absence of any psychological, familial, sociological or geographical condition potentially hampering compliance with the study protocol and follow-up schedule; those conditions should be discussed with the patient before registration in the trial
* The effects of RO4929097on the developing human fetus at the recommended therapeutic dose are unknown; for this reason and because Notch signal pathway inhibitors are known to cause interruption of the embryonic signaling pathway and may lead to serious or life-threatening birth defects, including brain deformities, facial malformation, heart problems, or abnormal organs; if women of childbearing potential do not abstain\* from sexual activity they must use two forms of contraception (i.e., barrier contraception and one other method of contraception) at least 4 weeks prior to study entry; women of childbearing potential can either be abstinent or use two forms of contraception for the duration of study participation and for at least 12 months post-treatment; should a woman become pregnant or suspect she is pregnant while she is participating in this study and for 12 months after study participation, the patient should inform the treating physician immediately

  * Abstinent from sexual activity at least 4 weeks prior to study entry
  * Pregnancy Testing: Women of childbearing potential are required to have a negative serum pregnancy test (with a sensitivity of at least 25 mIU/mL) within 10-14 days and within 24 hours prior to the first dose of RO4929097 (serum or urine); a pregnancy test (serum or urine) will be administered every 4 weeks if their menstrual cycles are regular or every 2 weeks if their cycles are irregular while on study within the 24-hour period prior to the administration of RO4929097; a positive urine test must be confirmed by a serum pregnancy test; prior to dispensing RO4929097, the investigator or clinical staff must confirm and document the patient's use of two contraceptive methods or abstinence, dates of negative pregnancy test, and confirm the patient's understanding of the potential of RO4929097 to cause serious or life-threatening birth defects
  * Female patients of childbearing potential are defined as follows:

    * Patients with regular menses
    * Patients, after menarche with amenorrhea, irregular cycles, or using a contraceptive method that precludes withdrawal bleeding
    * Women who have had tubal ligation
  * Female patients may be considered to NOT be of childbearing potential for the following reasons:

    * The patient has undergone total abdominal hysterectomy with bilateral salpingo-oophorectomy or bilateral oophorectomy
    * The patient is medically confirmed to be menopausal (no menstrual period) for 24 consecutive months
    * Pre-pubertal females; the parent or guardian of young female patients who have not yet started menstruation should verify that menstruation has not begun; if a young female patient reaches menarche during the study, then she is to be considered as a woman of childbearing potential from that time forward
* Ability to understand and the willingness to sign a written informed consent document

Exclusion Criteria:

* Patients may not be receiving any other investigational agents
* Patients with known brain metastases should be excluded from this clinical trial because of their poor prognosis and because they often develop progressive neurologic dysfunction that would confound the evaluation of neurologic and other adverse events
* History of allergic reactions attributed to compounds of similar chemical or biologic composition to RO4929097 or other agents used in the study
* Patients taking medications with narrow therapeutic indices that are metabolized by cytochrome P450 (CYP450), including warfarin sodium (Coumadin ®) are ineligible
* Preclinical studies indicate that RO4929097 is a substrate of CYP3A4 and inducer of CYP3A4 enzyme activity; caution should be exercised when dosing RO4929097 concurrently with CYP3A4 substrates, inducers, and/or inhibitors; patients who are taking concurrent medications that are strong inducers, inhibitors or substrates of CYP3A4 should be switched to alternative medications to minimize any potential risk; if such patients cannot be switched to alternative medications, they will be ineligible to participate in this study
* Patients with malabsorption syndrome, bowel obstruction, or other condition that would interfere with intestinal absorption; patients must be able to swallow tablets
* Patients who are serologically positive for Hepatitis A, B or C, or have a history of liver disease, other forms of hepatitis or cirrhosis are ineligible
* Patients with uncontrolled hypocalcemia, hypomagnesemia, hyponatremia, hypophosphatemia or hypokalemia defined as less than the lower limit of normal for the institution, despite adequate electrolyte supplementation are excluded from this study
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia other than chronic, stable atrial fibrillation, or psychiatric illness/social situations that would limit compliance with study requirements
* Pregnant women are excluded from this study because RO4929097 is a Notch pathway inhibiting agent with the potential for serious or life-threatening birth defects or abortifacient effects; because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with RO4929097, breastfeeding should be discontinued if the mother is treated with RO4929097; these potential risks may also apply to other agents used in this study
* Known HIV-positive patients on combination antiretroviral therapy are ineligible because of the potential for pharmacokinetic interactions with RO4929097; in addition, these patients are at increased risk of lethal infections when treated with marrow-suppressive therapy; appropriate studies will be undertaken in patients receiving combination antiretroviral therapy when indicated
* Cardiovascular (within 7 days prior to start of study treatment): baseline QTc \> 470 msec (female), QTc \> 450 msec (male)

  * History of risk factors for QT interval prolongation, including, but not limited to family or personal history of long QT syndrome, recurrent syncope without known etiology or sudden unexpected death
  * History of torsade de pointes or other significant cardiac arrhythmias or the need for concomitant meds with known potential to prolong QT interval or antiarrhythmics