The Department of Housing and Community Affairs Housing Code Enforcement Unit investigates complaints regarding the MAINTENANCE or deteriorated CONDITION of a fence, shed or other structure.  If this is a complaint about building without a permit or building too close to property line, please see the topic entitled: 
"FILE COMPLAINT WITH DEPARTMENT OF PERMITTING SERVICES".  The property owner is responsible for maintaining fences and other structures on residential property.  See Section 26-9(a)(1) of Chapter 26, Housing and Building and Maintenance Standards on the attached link for the Code reference. 
To file a complaint regarding a fence, shed, retaining wall, falling down/ in disrepair, please see the topic entitled  "HOUSING COMPLAINTS" , or click on the second link below.