Axitinib With or Without Anti-OX40 Antibody PF-04518600 in Treating Patients With Metastatic Kidney Cancer

Inclusion Criteria:

* Willing and able to provide informed consent
* Histological confirmation of renal cell carcinoma (RCC) with a predominantly (\> 50%) clear cell component
* Metastatic RCC
* Must have had a nephrectomy (radical or partial) and must provide the cell block from the nephrectomy
* Measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST)1.1 criteria
* Must have progression of disease within 6 months of study enrollment after treatment with only one of the following:

  * Two prior lines of therapy: a VEGF inhibitor (other than axitinib), followed by a single agent PD-1/PDL-1 antibody, or
  * One prior line of therapy: combination of a VEGF inhibitor (other than axitinib) AND a PD1/PDL1 antibody, or
* Additional prior systemic treatments not allowed
* Must agree to a fresh core or excisional biopsy from a metastatic site within a 12-week window prior to enrollment; if such a biopsy is already available, cell blocks must be provided; (Note: fine needle aspiration \[FNA\] and bone metastases samples are not acceptable for submission); specimens from the nephrectomy and fresh biopsy must be received and assessed for adequacy of tissue by the Data Coordinating Center (DCC) (University of Southern California \[USC\]) prior to randomization
* Zubrod performance status of =\< 2
* Women of childbearing potential must use method(s) of contraception; the individual methods of contraception should be determined in consultation with the treating physician or investigator
* Women of childbearing potential must have a negative serum pregnancy test within 24 hours prior to the administration of the investigational product; female patients who are not of childbearing potential as defined below, are eligible to be included (ie, meet at least one of the following criteria):

  * Have undergone a documented hysterectomy and/or bilateral oophorectomy
  * Have medically confirmed ovarian failure; or
  * Achieved postmenopausal status, defined as follows: cessation of regular menses for at least 12 consecutive months with no alternative pathological or physiological cause; a serum follicle stimulating hormone (FSH) level within the laboratory's reference range for postmenopausal women
* Women must not be breastfeeding
* Men who are sexually active with women of childbearing potential must use any contraceptive method with a failure rate of less than 1% per year
* Contraception should be continued using two highly effective methods for a period of 90 days
* Serum creatinine =\< 1.5 x upper limit of normal (ULN) OR creatinine clearance (CrCl) \>= 40 mL/min (measured or calculated using the Cockcroft-Gault formula) using actual weight (ideal or adjusted weights are unacceptable)
* White blood cells (WBC) \>= 2000/uL
* Neutrophils \>= 1500/uL
* Platelets \>= 100x10\^3/uL
* Hemoglobin \>= 9g/dL
* Aspartate aminotransferase (AST) =\< 3 x ULN
* Alanine aminotransferase (ALT) =\< 3 x ULN
* Bilirubin =\< 1.5 x ULN

Exclusion Criteria:

* Patients with known symptomatic brain metastases requiring systemic corticosteroids; patients with previously diagnosed brain metastases are eligible if they have completed their treatment and have recovered from the acute effects of radiation therapy or surgery prior to the start of study medication, have discontinued corticosteroid treatment for these metastases for at least 4 weeks and are neurologically stable; mild neurological deficit is allowed, if it does not interfere with the ability to judge the safety on the trial
* Prior treatment with an mTOR inhibitor (including, but not limited to, everolimus, temsirolimus, sirolimus, and ridaforolimus)
* Prior treatment with axitinib
* History of or active autoimmune disorders (including but not limited to: Crohn's disease, rheumatoid arthritis, scleroderma, systemic lupus erythematosus, Grave's disease) and other conditions that compromise or impair the immune system
* Active bacterial, fungal or viral infection including hepatitis B (HBV), hepatitis C (HCV), known human immunodeficiency virus (HIV) or acquired immunodeficiency syndrome (AIDS) -related illness
* Any condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalent) or other immunosuppressive medications within 14 days prior to the first dose of study drug; inhaled steroids and adrenal replacement steroid doses \> 10 mg daily prednisone equivalent are permitted in the absence of active autoimmune disease
* Uncontrolled adrenal insufficiency
* Any known active chronic liver disease
* Prior malignancy active within the previous 3 years except for locally curable cancers that have been apparently cured, such as basal or squamous cell skin cancer, superficial bladder cancer, or carcinoma in situ of the prostate, cervix or breast
* Known medical condition (e.g., a condition associated with diarrhea or acute diverticulitis) that, in the investigator's opinion, would increase the risk associated with study participation or study drug administration or interfere with the interpretation of safety results
* Prior treatment with an anti-CD137, or OX40 antibody, or any other antibody or drug specifically targeting T-cell co-stimulation or checkpoint pathways except anti-PD1, anti-PDL1/2 antibodies or ipilimumab
* Major surgery less than 6 weeks prior to the first dose of study drug; minor surgery less than 4 weeks prior to the first dose of study drug
* Anti-cancer therapy less than 6 weeks prior to the first dose of study drug (less than 28 days for bevacizumab) or palliative, focal radiation therapy less than 14 days prior to the first dose of study drug
* Presence of toxicities attributed to prior therapy other than alopecia that have not resolved to grade 1 (National Cancer Institute \[NCI\] Common Terminology Criteria for Adverse Events \[CTCAE\] version 4) or baseline before administration of study drug
* History of grade 3 or higher immune-mediated adverse event (including AST/ALT elevations that where considered drug related and cytokine release syndrome) that was considered related to prior immune-modulatory therapy (e.g., checkpoint inhibitors, costimulatory agents etc.) or any grade immune-related adverse events (AEs) that required immune suppressive therapy
* Patients with intolerance to or who have had a severe (\>= grade 3) allergic or anaphylactic reaction to antibodies or infused therapeutic proteins, or patients who have had a severe allergic or anaphylactic reaction to any of the substances included in the investigational product (including excipients)
* Patients with a previous history of adriamycin treatment and are at risk of cardiac failure (New York Heart Association \[NYHA\] class II or above)
* Any one of the following currently or in the previous 6 months:

  * Myocardial infarction
  * Congenital long QT syndrome
  * Torsade's de points
  * Arrhythmias (including sustained ventricular tachyarrhythmia and ventricular fibrillation), and left anterior hemiblock (bifascicular block)
  * Unstable angina, coronary/peripheral artery bypass graft
  * Symptomatic congestive heart failure (congestive heart failure \[CHF\] New York Heart Association class III or IV)
  * Cerebrovascular accident, transient ischemic attack or symptomatic pulmonary embolism or other clinical significant episode of thrombo-embolic disease (Cases must be discussed in detail with study chair to judge eligibility; anticoagulation (heparin only, no vitamin-K antagonists or factor Xa inhibitors) will be allowed if indicated)
  * Ongoing cardiac dysrhythmias of NCI CTCAE grade \>= 2, atrial fibrillation of any grade, or QT correction using Fridericia's correction formula (QTcF) interval \> 470 msec at screening (except in case of right bundle branch block, these cases must be discussed with sponsor's medical monitor)
* Other severe acute or chronic medical or psychiatric condition, including recent (within the past year) or active suicidal ideation or behavior, or laboratory abnormality that may increase the risk associated with study participation or investigational product administration or may interfere with the interpretation of study results and, in the judgment of the investigator, would make the patient inappropriate for entry into this study
* Concurrent use of any medications or substances; these include steroids as they may interfere with PF-04518600 (OX40 Ab); also strong CYP3A4/5 inhibitors should be avoided (e.g., ketoconazole, itraconazole, clarithromycin, atazanavir, indinavir, nefazodone, nelfinavir, ritonavir, saquinavir, telithromycin, and voriconazole)
* Presence of a malabsorption syndrome, gastrointestinal disorder, or gastrointestinal surgery that could affect the absorption of axitinib
* History of severe hypersensitivity reaction to any monoclonal antibody
* Women who are pregnant or breastfeeding
* Women with a positive pregnancy test
* Prisoners or patients who are involuntarily detained