Login

Recent searches

No recent searches

Search options

Only available when logged in.
fosstodon.org is one of the many independent Mastodon servers you can use to participate in the fediverse.
Fosstodon is an invite only Mastodon instance that is open to those who are interested in technology; particularly free & open source software. If you wish to join, contact us for an invite.

Administered by:

Server stats:

10K
active users

fosstodon.org: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

#selfsupervisedlearning

0 posts0 participants0 posts today
Public
Hacker News

Self-Supervised Learning from Images with JEPA

arxiv.org/abs/2301.08243

arXiv logo
arXiv.orgSelf-Supervised Learning from Images with a Joint-Embedding Predictive ArchitectureThis paper demonstrates an approach for learning highly semantic image representations without relying on hand-crafted data-augmentations. We introduce the Image-based Joint-Embedding Predictive Architecture (I-JEPA), a non-generative approach for self-supervised learning from images. The idea behind I-JEPA is simple: from a single context block, predict the representations of various target blocks in the same image. A core design choice to guide I-JEPA towards producing semantic representations is the masking strategy; specifically, it is crucial to (a) sample target blocks with sufficiently large scale (semantic), and to (b) use a sufficiently informative (spatially distributed) context block. Empirically, when combined with Vision Transformers, we find I-JEPA to be highly scalable. For instance, we train a ViT-Huge/14 on ImageNet using 16 A100 GPUs in under 72 hours to achieve strong downstream performance across a wide range of tasks, from linear classification to object counting and depth prediction.
#HackerNews#SelfSupervisedLearning#JEPA
Replied in thread
Public
Fabrice Tshimanga

@joss Happy to share this mini-paper and library I'm co-authoring.
Thanks to Federico, Andrea, Paolo and Manfredo, unfortunately none of them here (yet).
We're working on #deeplearning applications to #neuroscience and #EEG is very different from the data Big Tech usually approaches, so results and models are quite different there...
But we believe #selfsupervisedlearning is a great idea and we'd like for researchers to come play with it 👨🏾‍💻🧠

Public
Fabrice Tshimanga

The preprint of our lab's library for #selfsupervisedlearning on #eeg data is out!
Check it at arxiv.org/abs/2401.05405

The repo (under review with the preprint for the amazing @joss ) is at github.com/MedMaxLab/selfEEG

If you want to try deep learning and EEG, if you have lots of data, but supervised learning is difficult or ineffective for your target task, you might want to experiment with self-supervised learning as popularized for #transformers and vision models!
Techniques such as MoCo, SimCLR are already implemented, and eeg augmentations can be used and further customized. If you do not know how to come up with architectures, don't worry! A model zoo is there 👨🏾‍💻🧠

arXiv.orgSelfEEG: A Python library for Self-Supervised Learning in ElectroencephalographySelfEEG is an open-source Python library developed to assist researchers in conducting Self-Supervised Learning (SSL) experiments on electroencephalography (EEG) data. Its primary objective is to offer a user-friendly but highly customizable environment, enabling users to efficiently design and execute self-supervised learning tasks on EEG data. SelfEEG covers all the stages of a typical SSL pipeline, ranging from data import to model design and training. It includes modules specifically designed to: split data at various granularity levels (e.g., session-, subject-, or dataset-based splits); effectively manage data stored with different configurations (e.g., file extensions, data types) during mini-batch construction; provide a wide range of standard deep learning models, data augmentations and SSL baseline methods applied to EEG data. Most of the functionalities offered by selfEEG can be executed both on GPUs and CPUs, expanding its usability beyond the self-supervised learning area. Additionally, these functionalities can be employed for the analysis of other biomedical signals often coupled with EEGs, such as electromyography or electrocardiography data. These features make selfEEG a versatile deep learning tool for biomedical applications and a useful resource in SSL, one of the currently most active fields of Artificial Intelligence.
Public
Low Rank Jack

Update on my joint work with @JulianTachella on "#Learning to Reconstruct Signals From Binary Measurements" on arXiv. #RandomProjection #onebit #SelfSupervisedLearning arxiv.org/abs/2303.08691

We brought several improvements on the proofs and the bounds allowing us to determine from how many binarized (random) projections, only, one can learn, up to a controlled identification error, a low-complexity space (with small box dimemsion). Moreover, a practical #selfsupervised scheme, SSBM, run over real datasets of images, enables to learn a reconstruction algorithm from those same binary observations (without access to the original images and on par with supervised alternatives), implicitly confirming the encoding of a good estimate of the image set.

arXiv.orgLearning to Reconstruct Signals From Binary MeasurementsRecent advances in unsupervised learning have highlighted the possibility of learning to reconstruct signals from noisy and incomplete linear measurements alone. These methods play a key role in medical and scientific imaging and sensing, where ground truth data is often scarce or difficult to obtain. However, in practice, measurements are not only noisy and incomplete but also quantized. Here we explore the extreme case of learning from binary observations and provide necessary and sufficient conditions on the number of measurements required for identifying a set of signals from incomplete binary data. Our results are complementary to existing bounds on signal recovery from binary measurements. Furthermore, we introduce a novel self-supervised learning approach, which we name SSBM, that only requires binary data for training. We demonstrate in a series of experiments with real datasets that SSBM performs on par with supervised learning and outperforms sparse reconstruction methods with a fixed wavelet basis by a large margin.
Public *
Low Rank Jack

From Julián Tachella @JulianTachella, posted on "Chi":

📰""Learning to reconstruct signals from binary measurements alone"📰

We present theory + a #selfsupervised approach for learning to reconstruct incomplete (!) and binary (!) measurements using the binary data itself. See the first figure and its alt-text.

arxiv.org/abs/2303.08691
with @lowrankjack
---

The theory characterizes

- the best approximation of a set of signals from incomplete binary observations
- its sample complexity
- complements existing theory for signal recovery from binary measurements

See the third figure and its alt-text.
---

The proposed self-supervised algorithm obtains performances on par with supervised learning and outperforms standard reconstruction techniques (such as binary iterative hard thresholding)

See the second figure and its alt-text.

---

Code based on the deepinverse library is available at github.com/tachella/ssbm

Check out the paper for more details!

On this figure, a description of a self-supervised method for learning an image reconstruction algorithm (taking as input its binary observations) when one has access only to binary observations of many images belonging to the same, low complexity signal set. The figure shows a sensing device on the left acquiring numerous binary observations that are feeding either a linear inversion method on a top branch (such as a pseudo inverse associated with the sensing model before binarisation) with poor image reconstruction quality shown on the top right of the figure, or, on a bottom branch, a neural network model learned from binary observations alone, by promoting consistency with the binary sensing model (more information in the paper). This second branch achieves higher quality in the estimated images, as shown on the bottom right of the figure.
On this figure, a table of images with 5 rows and 10 columns showing 10 images of the FashionMnist dataset (on the last row) reconstructed from their binary observations for 4 different algorithms, one per row, for 10 images of the dataset (such as a shoe, a shirt, ...). The 4 algorithms are: linear inversion, binary iterative hard thresholding (or BIHT, proposed in one bit compressive sensing), the proposed method using self-supervision, and a supervised method having access to the initial images to learn the reconstruction method. The proposed approach provides better quality than BIHT with a wavelet prior and the linear approach, and it is close to the image quality of the supervised method.
On this figure, a description of a self-supervised method for learning an image reconstruction algorithm (taking as input its binary observations) when one has access only to binary observations of many images belonging to the same, low complexity signal set. The figure shows a sensing device on the left acquiring numerous binary observations that are feeding either a linear inversion method on a top branch (such as a pseudo inverse associated with the sensing model before binarisation) with poor image reconstruction quality shown on the top right of the figure, or, on a bottom branch, a neural network model learned from binary observations alone, by promoting consistency with the binary sensing model (more information in the paper). This second branch achieves higher quality in the estimated images, as shown on the bottom right of the figure.
#SelfSupervisedLearning#CompressiveSensing#Quantization
ExploreLive feeds
About