Language selection

Government of Canada / Gouvernement du Canada

Search

Data-efficient training of autoencoders for mildly non-linear problems

From National Research Council Canada

Linkhttps://s2d-olad.github.io/posters/submission-13.pdf
AuthorSearch for: 1; Search for: 1; Search for: 2; Search for: ; Search for: 2; Search for: 2; Search for: 2; Search for: 2
Affiliation
  1. National Research Council of Canada. Digital Technologies
  2. National Research Council of Canada. Advanced Electronics and Photonics
FormatText, Poster
ConferenceNinth International Conference on Learning Representations 2021, May 3-7, 2021, Virtual Only
Abstract
Principal Component Analysis (PCA) provides reliable dimensionality reduction (DR) when data possesses linear properties even for small datasets. However, faced with data that exhibits non-linear behaviour, PCA cannot perform optimally as compared to non-linear DR methods such as AutoEncoders. By contrast, AutoEncoders typically require much larger datasets for training than PCA. This data requirement is a critical impediment in applications where samples are scarce and expensive to come by. One such area is nanophotonics component design where generating a single data point might involve running optimization methods that use computationally demanding solvers. We propose Guided AutoEncoders (G-AE), standard AutoEncoders initialized using a numerically stable procedure to replicate PCA behaviour before training. Our results show this approach yields a marked reduction in the data size requirements for training the network along with gains in capturing non-linearity during dimensionality reduction and thus performing better than PCA alone.
Publication date
PublisherInternational Conference on Learning Representations
Other format
LanguageEnglish
Peer reviewedNo
Export citationExport as RIS
Report a correctionReport a correction (opens in a new tab)
Record identifier8af970a3-6267-4ab6-8e0a-f21216975fdd
Record created2022-05-27
Record modified2023-07-27
Date modified: