Paclitaxel With or Without Cixutumumab as Second-Line Therapy in Treating Patients With Metastatic Esophageal Cancer or Gastroesophageal Junction Cancer

Inclusion Criteria:

* Life expectancy \>= 12 weeks
* Women must not be pregnant or breast-feeding due to potential harm to fetus from cixutumumab (IMC-A12) and paclitaxel; all females of childbearing potential must have a blood test or urine study within 48 hours prior to registration to rule out pregnancy
* Women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method or birth control; abstinence) for the duration of study therapy and for 3 months after the last dose of cixutumumab (IMC-A12); should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately
* Patients must have measurable disease
* Patients must have metastatic disease of the esophagus or gastroesophageal junction

  * Histologic, cytologic or radiologic documentation of metastatic squamous cell carcinoma or adenocarcinoma of the esophagus or gastroesophageal junction; radiologic, endoscopic, histologic or cytologic evidence of locally recurrent or locally residual (post-resection) disease is also permitted
  * For the purposes of this study, undifferentiated adenocarcinomas and adenosquamous tumors will be considered as adenocarcinomas; in addition, tumors involving the gastroesophageal junction will be defined by the Siewert classification
* Patients with gastroesophageal junction tumors who are eligible:

  * Adenocarcinoma of the esophageal junction (AEG) Type I: adenocarcinoma of the distal esophagus which usually arises from an area with specialized intestinal metaplasia of the esophagus, i.e., Barrett's esophagus, and may infiltrate the esophagogastric junction from above
  * AEG Type II: true carcinoma of the cardia arising from the cardiac epithelium or short segments with intestinal metaplasia at the esophagogastric junction
* Patients with gastroesophageal junction tumors who are NOT eligible:

  * AEG Type III: subcardial gastric carcinoma which infiltrates the esophagogastric junction and distal esophagus from below
* Patients must have received and progressed on one and only one line of prior systemic therapy for esophagus or esophagogastric cancer; this could have included one regimen for metastatic disease, or one regimen with radiotherapy for initially locally advanced disease; prior radiation therapy is permitted

  * If patients progress or recur within 6 months of neoadjuvant/adjuvant therapy, this will be considered one line of therapy; for patients progressing or recurring more than 6 months after neoadjuvant/adjuvant therapy, they will need to receive one line of therapy for recurrent disease to be eligible
  * If patients receive one regimen in which a chemotherapy agent is dropped for toxicity without progression, this treatment will be considered one line of therapy; however, substitution or addition of a new agent will be considered a second line of therapy
* Eastern Cooperative Oncology Group (ECOG) performance status 0-2
* Leukocytes \> 3,000/mcL
* Absolute neutrophil count \>= 1,500/mcL
* Hemoglobin \>= 9 g/dL
* Platelets \>= 100,000/mcL
* Total bilirubin =\< institutional upper limit of normal (ULN)
* Aspartate transaminase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine transaminase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 X institutional ULN
* Creatinine =\< 1.5 X institutional ULN or creatinine clearance \>= 60 mL/min/1.73m\^2 for patients with creatinine levels above institutional normal
* Patients must have fasting serum glucose =\< 160 mg/dL (8.8 mmol/L) or =\< ULN, and hemoglobin A1C =\< 7% (0.07 International System of Units \[SI units\]) within 14 days of registration; if baseline nonfasting glucose =\< 160 mg/dL (8.8 mmol/L), fasting glucose measurement is not required
* Registration no fewer than 28 days from last chemotherapy
* A "currently active" second malignancy other than non-melanoma skin cancers are not to be registered; patients are not considered to have a "currently active" malignancy if they have completed therapy and are considered by their physician to be at less than 30% risk of relapse

Exclusion Criteria:

* Patients have received prior taxane or anti-insulin growth factor receptor (IGFR) therapy
* Patients must not have any of the following conditions:

  * Poorly controlled diabetes mellitus; patients with a history of diabetes mellitus are allowed to participate, provided that their blood glucose is within normal range (fasting glucose =\< 160 mg/dL \[8.8 mmol/L\] or below the ULN and hemoglobin A1C =\< 7% \[0.07 SI units\]) and that they are on a stable dietary or therapeutic regimen for this condition
  * Recent major surgery, hormonal therapy (other than replacement) or chemotherapy, within 4 weeks prior to entering the study or those who have not recovered from adverse events
  * History of allergic reactions attributed to compounds of similar chemical or biologic composition to cixutumumab (IMC-A12)
  * Psychiatric illness that would prevent the patient from giving informed consent
* Medical conditions such as active/uncontrolled infection (including HIV) or cardiac disease that would make this protocol unreasonably hazardous for the patient in the opinion of the treating physician; cardiac disease may include uncontrolled high blood pressure, unstable angina, or serious uncontrolled cardiac arrhythmia