Sirolimus and Vismodegib in Treating Patients With Solid Tumors or Pancreatic Cancer That is Metastatic or Cannot Be Removed By Surgery

Inclusion Criteria:

* COHORT I (DOSE ESCALATION): histologic proof of cancer that is now unresectable, not amenable to any other standard therapies, or patient refuses standard therapy
* COHORT II (MTD): metastatic adenocarcinoma of the pancreas and tumor amenable to biopsies; prior systemic treatment for metastatic disease is allowed
* Absolute neutrophil count (ANC) =\> 1500/uL
* Platelet \>= 100,000/uL
* Total bilirubin =\< upper limit of normal (ULN) Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) =\< 3 x ULN
* Creatinine =\< 1.5 x ULN
* Hemoglobin \>= 9.0 g/dL
* Prothrombin time (PT)/international normalized ratio (INR) \< 1.25 x ULN (Cohort II \[MTD\] only)
* Cholesterol \< Common Terminology Criteria for Adverse Events (CTCAE) grade 3
* Triglycerides \< CTCAE grade 2
* Magnesium \>= lower limit of normal (LLN) and =\< ULN
* Ability to provide informed consent
* Willing to return to Mayo Clinic for follow up
* Life expectancy \>= 12 weeks
* Cohort II (MTD) only - Translational Research: Willing to provide the biologic specimens as required by the protocol; Note: this is part of the mandatory translational research component
* Women of childbearing potential only: Negative serum pregnancy test done =\< 7 days prior to registration; NOTE: female subjects who are pregnant or nursing are excluded from this study; there is no specific mitigation strategy for vismodegib toxicity; however, male patients should be made aware of it during the consent process; although this effect is expected to be reversible with discontinuation of dosing, long-term effects on male fertility cannot be excluded at this time
* Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0, 1 or 2
* Able to swallow or have medication administered through a G-tube and absorb the medication
* Participant agrees to use acceptable form of contraception during the study and for up to 7 months after last study drug dose
* Acceptable forms of contraception:

  * Latex condom (always used with spermicide)
  * Diaphragm (always used with spermicide)
  * Cervical cap (always used with spermicide)
* Acceptable forms of secondary contraception, when used along with a barrier method:

  * Hormonal contraception methods, including pills, patches, rings, or injections except progestin-only containing pills (i.e. "Mini-pill")
  * Tubal ligation
  * Partner's vasectomy
  * Intrauterine device (non-progesterone T)
  * Vaginal sponge (containing spermicide)
* Other acceptable forms:

  * 100% commitment to abstinence
* Unacceptable forms of contraception for women of childbearing potential:

  * Oral contraception containing progestins only
  * Intrauterine device (IUD) progesterone T
  * Female condom
  * Natural family planning (rhythm method) or breastfeeding
  * Fertility awareness
  * Withdrawal
  * Cervical shield
* Willing not to smoke
* Willing to complete a pill diary each day

Exclusion Criteria:

* COHORT I (DOSE ESCALATION): Known standard therapy for the patient's disease that is potentially curative or definitely capable of extending life expectancy
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, diabetes mellitus or psychiatric illness/social situations that would limit compliance with study requirements
* Any of the following prior therapies:

  * Chemotherapy =\< 4 weeks prior to registration
  * Mitomycin C/nitrosoureas =\< 6 weeks prior to registration
  * Immunotherapy =\< 4 weeks prior to registration
  * Biologic therapy =\< 4 weeks prior to registration
  * Radiation therapy =\< 4 weeks prior to registration
  * Radiation to \> 25% of bone marrow
* Failure to fully recover from acute, reversible effects of prior chemotherapy regardless of interval since last treatment
* New York Heart Association classification III or IV
* Uncontrolled Seizure Disorder
* Central nervous system (CNS) metastases if not stable for at least 2-3 months based on imaging, clinical assessment, use of steroids, or seizure disorder
* Any of the following:

  * Pregnant women
  * Nursing women
  * This study involves an investigational agent whose genotoxic, mutagenic and teratogenic effects on the developing fetus and newborn are unknown
* Other concurrent chemotherapy, immunotherapy, radiotherapy, any ancillary therapy considered investigational (utilized for a non-Food and Drug Administration \[FDA\]-approved indication and in the context of a research investigation) or receiving any other investigational agent which would be considered as a treatment for the primary neoplasm
* Receiving any medications or substances that are strong or moderate inhibitors of cytochrome P450 3A4 (CPY450 3A4); use of the following strong or moderate inhibitors are prohibited:
* Strong Inhibitors of CYP3A4: Indinavir, Nelfinavir, Ritonavir, Clarithromycin, Itraconazole, Ketoconazole, Nefazodone, Saquinavir, Telithromycin

  * Moderate Inhibitors of CYP3A4: Aprepitant, Erythromycin, Fluconazole, Grapefruit juice, Verapamil, Diltiazem
* Receiving any medications or substances that are inducers of CYP450 3A4

  * Inducers of CYP3A4: Efavirenz, Nevirapine, Carbamazepine, Modafinil, Phenobarbital, Phenytoin, Pioglitazone, Rifabutin, Rifampin, St. John's wort
* Receiving any medications or substances that are strong or moderate inhibitors of CYP450 CYP2C8

  * Strong or Moderate Inhibitors of CYP2C8: Gemfibrozil, Trimethoprim
* Receiving any medications or substances that are inducers of CYP450 2C8

  * Inducer of CYP2C8: Rifampin
* Receiving any medications or substances that are strong or moderate inhibitors of CYP450 CYP2C9

  * Strong or Moderate Inhibitors of CYP2C9: Fluconazole, Amiodarone
* Receiving any medications or substances that are inducers of CYP450 2C9

  * Inducers of CYP2C9: Rifampin, Secobarbital
* Immunocompromised patients (other than that related to the use of corticosteroids) including patients receiving highly active antiretroviral therapy (HAART) treatment
* Active other malignancy, excepting non-melanotic skin cancer or carcinoma-in situ (e.g. of cervix, breast prostate); if there is a history of prior malignancy, they must not be receiving other specific treatment (other than hormonal therapy) for their cancer
* History of myocardial infarction =\< 6 months, or congestive heart failure requiring use of ongoing maintenance therapy for life-threatening ventricular arrhythmias
* Abnormalities of the cornea based on history (e.g., dry eye syndrome, Sjogren's syndrome), congenital abnormality (e.g., Fuch's dystrophy), abnormal slit-lamp examination using a vital dye (e.g., fluorescein, Bengal Rose), and/or an abnormal corneal sensitivity test (Schirmer test or similar tear production test)
* Prior therapy with a hedgehog inhibitor