We report the discovery of diffuse extended Ly-alpha emission from redshift 3.1 to 4.5, tracing cosmic web filaments on scales of 2.5-4 comoving Mpc. These structures have been observed in overdensities of Ly-alpha emitters in the MUSE Extremely Deep Field, a 140 hour deep MUSE observation located in the Hubble Ultra Deep Field. Among the 22 overdense regions identified, 5 are likely to harbor very extended Ly-alpha emission at high significance with an average surface brightness of 5×10−20ergs−1cm−2arcsec−2. Remarkably, 70% of the total Ly-alpha luminosity from these filaments comes from beyond the circumgalactic medium of any identified Ly-alpha emitters. Fluorescent Ly-alpha emission powered by the cosmic UV background can only account for less than 34% of this emission at z≈3 and for not more than 10% at higher redshift. We find that the bulk of this diffuse emission can be reproduced by the unresolved Ly-alpha emission of a large population of ultra low luminosity Ly-alpha emitters (<1040ergs−1), provided that the faint end of the Ly-alpha luminosity function is steep (α⪅−1.8), it extends down to luminosities lower than 1038−1037ergs−1 and the clustering of these Ly-alpha emitters is significant (filling factor <1/6). If these Ly-alpha emitters are powered by star formation, then this implies their luminosity function needs to extend down to star formation rates <10−4M⊙yr−1. These observations provide the first detection of the cosmic web in Ly-alpha emission in typical filamentary environments and the first observational clue for the existence of a large population of ultra low luminosity Ly-alpha emitters at high redshift.