Chao Cao*, Tong Chen*, Nan Zhao*, Minheng Chen, Michael Qu, Zeyu Zhang, Xiao Shi, Xiang Li, Tianming Liu, Lu Zhang · *equal contribution IEEE International Symposium on Biomedical Imaging (ISBI), 2026
The GS-Net pipeline: (a) split the cortex into gyri and sulci, (b) erode both into thin skeletons, (c) march and trim them into GyralNet and SulcalNet, (d) join the two networks and identify the landmarks.
Puts the brain’s two folding patterns, gyri and sulci, into one network, so fine-scale folding landmarks can be studied the way brain networks are.
Problem
Brain networks usually take whole regions as nodes, which hides the fine-scale landmarks inside them, such as gyral peaks and sulcal fundi. Gyral and sulcal patterns have also been modelled separately.
Approach
Segment the cortical surface into gyri and sulci on the sphere, erode both sides into skeletons, build GyralNet and SulcalNet by tree marching and trimming, then connect the free sulci to GyralNet junctions. The joint network has three kinds of nodes: gyri conjunctions, sulci–gyri conjunctions and sulci conjunctions.
Findings
Over 95% of free sulci end directly at a GyralNet junction, so almost all folding fits one network. Using several surface measures instead of sulcal depth alone removes spurious 3-hinge detections and cuts the runtime from hours to seconds per subject.
Data
1,600+ T1-weighted scans from 34 gestational weeks to old age: HCP young adults (1,064), ADNI elderly adults (282) and Alzheimer’s patients (200), and infant cohorts at 34 weeks, 6 months and 2 years.
@inproceedings{cao2026gsnet,
title = {Gyral-Sulcal-Net: An Integrated Network Representation of Brain Folding Patterns},
author = {Cao, Chao and Chen, Tong and Zhao, Nan and Chen, Minheng and Qu, Michael and Zhang, Zeyu and Shi, Xiao and Li, Xiang and Liu, Tianming and Zhang, Lu},
booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
year = {2026},
note = {arXiv:2601.08818}
}
Chao Cao*, Jiale Cheng*, Minheng Chen, Jing Zhang, Tianming Liu, Lu Zhang, Dajiang Zhu, Gang Li · *equal contribution IEEE International Symposium on Biomedical Imaging (ISBI), 2026
Deep-GyralNet: the cortical surface is mapped to a sphere, a Spherical U-Net predicts gyral crests, refinement makes them continuous, and GyralNet with its 3-hinge gyri is generated.
Replaces the slow geometric pipeline that extracts GyralNet and 3-hinge gyri with a learned one: about 17× faster, recovering 98% of the reference network.
Problem
3-hinge gyri, where three gyri meet, are hubs of the cortex; joined along gyral crests they form GyralNet. Extracting it takes a long chain of geometric steps, which limits large-sample studies.
Approach
Map each surface to a sphere and let a Spherical U-Net label gyral-crest vertices from convexity and curvature, trained with a focal loss plus a Connectivity-Aware Path Enforcement loss that keeps ridges continuous. Confidence-guided geodesic skeletonization and minimum-spanning-tree refinement then give a topologically valid GyralNet; 3-hinge gyri are its three-way junctions.
Findings
On the Human Connectome Project with 6-fold cross-validation: recall 98%, precision 94%, F1 96% and Dice 72% across both hemispheres. Average runtime falls from 237 s to 14 s. Removing either loss term lowers Dice.
Data
Human Connectome Project, T1-weighted MRI of 1,098 subjects.
@inproceedings{cao2026deepgyralnet,
title = {Deep-GyralNet: Enabling Machine Learning in Gyral Folding Pattern Extraction on Cortical Surface},
author = {Cao, Chao and Cheng, Jiale and Chen, Minheng and Zhang, Jing and Liu, Tianming and Zhang, Lu and Zhu, Dajiang and Li, Gang},
booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
year = {2026},
doi = {10.1109/ISBI61048.2026.11515596}
}
Chao Cao, Xiaowei Yu, Lu Zhang, Tong Chen, Yanjun Lyu, Tianming Liu, Dajiang Zhu IEEE International Symposium on Biomedical Imaging (ISBI), 2024
An anchor 3-hinge gyrus (red) is matched in other brains: its anatomical embedding shortlists candidates (yellow), and the fiber bundles through each candidate decide which one corresponds.
Gives each 3-hinge gyrus one counterpart in every other brain by adding diffusion-MRI fiber connectivity to an anatomical embedding.
Problem
3-hinge gyri are found in each brain separately, so they have no built-in correspondence across subjects. Matching on anatomical embeddings alone often returns several candidates for one 3-hinge.
Approach
Shortlist candidates whose cortex2vector anatomical embedding is close to the anchor’s (cosine similarity ≥ 0.9), then compare the DTI fiber bundles passing through the anchor and each candidate as trace-maps, and keep the candidate with the most similar connectivity.
Findings
Trained on 565 HCP subjects (186,915 3-hinge gyri) and applied to the remaining 500. For the 171 anchors of a reference brain, the selected match stands clearly apart from the other candidates in connectivity similarity (p = 5.02 × 10⁻²³), turning one-to-many matches into one-to-one.
Data
Human Connectome Project, T1-weighted MRI and DTI of 1,065 subjects.
@inproceedings{cao2024enhancing,
title = {Enhancing Group-Wise Consistency in 3-Hinge Gyrus Matching via Anatomical Embedding and Structural Connectivity Optimization},
author = {Cao, Chao and Yu, Xiaowei and Zhang, Lu and Chen, Tong and Lyu, Yanjun and Liu, Tianming and Zhu, Dajiang},
booktitle = {IEEE International Symposium on Biomedical Imaging (ISBI)},
year = {2024},
doi = {10.1109/ISBI56570.2024.10635893}
}
Medical image models are usually trained on images and labels alone, and earlier eye-tracking work used a radiologist's gaze only as a static heatmap. GazeWorld treats a chest X-ray as a world and the radiologist's fixations as a path through it: a causal Transformer learns to predict, in feature space, the next region the expert will look at and the regions not yet visited, so the image encoder absorbs the order in which experts gather evidence. Gaze is needed only for training (3,032 MIMIC-Eye images). The refined encoder beat MedCLIP, MGCA and other baselines in every CheXpert, RSNA Pneumonia and SIIM-ACR linear-probe setting (CheXpert with 1% of labels: 78.4 vs 74.7 AUROC) and in zero-shot classification, which makes expert reading behaviour a cheap supervision signal for label-scarce medical imaging.
Vision Transformers decide where to attend from feature similarity alone, so a patch can attend to a region that merely looks alike but is clinically unrelated. DCMM-Transformer adds a learned community prior to attention: every patch gets soft memberships in latent communities (organ, lesion, background) and a degree term for hub regions such as a tumour core, and the resulting degree-corrected mixed-membership matrix is added to the attention logits. The model stays fully differentiable, where earlier stochastic-block-model masks were sampled and binary. Across chest X-ray, pneumothorax, mammography, fundus and ADNI connectivity data it raised average accuracy to 87.3%, from 83.6% for ViT and 83.1% for SBM-Transformer, with attention maps that stay on anatomically coherent regions.
Can a general-purpose LLM handle highly specialised clinical language, or does each hospital task still need a model fine-tuned on its own labelled data? The paper builds RadQNLI, a 9,838-pair radiology natural-language-inference benchmark from RadQA reports in MIMIC-III, and compares ChatGPT and GPT-4 (zero-shot, few-shot, chain-of-thought and a new instruction-response semantic alignment prompt) with open LLMs and five fine-tuned BERT-family models. With ten examples in the prompt, GPT-4 reached 0.84 accuracy, close to the BERT models fine-tuned on all 7,911 training pairs (0.85 on average), while the same models trained on ten examples averaged 0.52: evidence that one strong generic model can stand in for data-hungry task-specific ones.
A review of how large foundation models (language models, large vision models such as Segment Anything, and vision-language models) can be brought into medical imaging and healthcare. For each family it lays out roadmaps (expert-in-the-loop alignment, domain adaptation through prompts and adapters, knowledge graphs, federated multi-institution training, parameter-efficient fine-tuning), surveys uses from report generation to text-prompted segmentation and visual question answering, and names what still blocks clinical use: scarce paired data, privacy, expert-level accuracy, the long tail of rare diseases, bias and security. Its main message is that off-the-shelf general models fall short in medicine (zero-shot SAM trails specialised segmenters) and need medical data, adaptation and expert feedback.
A Unified Continuous Staging Framework for Alzheimer's Disease and Lewy Body Dementia via Hierarchical Anatomical FeaturesTong Chen, Minheng Chen, Jing Zhang, Yan Zhuang, Chao Cao, Xiaowei Yu, et al. (11 authors)
MICCAI 2025
Alzheimer's disease and Lewy body dementia share symptoms and pathology, so patients are often misdiagnosed or diagnosed late, and most models treat diagnosis as fixed classes rather than a progression. This framework fuses two views of each brain from T1 MRI and DTI, an atlas-based structural connectome and a finer graph of 3-hinge gyri (cortical folding hubs), with graph networks and anatomy-aware cross-attention, then learns class prototypes on a tree that orders subjects from healthy through MCI to AD and LBD. On 308 subjects (77 per group) it reached 61.7% four-class accuracy against 54.9–57.1% for connectome-only classifiers, and the trees place LBD near MCI and AD, a non-invasive, continuous picture of where a patient sits between the two dementias.
Lewy body disease has few labelled scans, while Alzheimer's datasets are large but come from other sites, scanners and protocols. TAT, a Vision Transformer on structural connectivity, learns from labelled ADNI subjects to diagnose an unlabelled LBD cohort: a patch-level discriminator scores how domain-invariant each connectivity patch is, those scores steer attention toward features that transfer, a global discriminator aligns whole-brain representations, and uncertain predictions are flagged as LBD, a class with no training labels. It is the first study of Alzheimer's-to-LBD domain adaptation; unlike closed-set baselines (ViT, TVT, SSRT) it can recognise LBD at all, which points to diagnosing rare diseases by borrowing from a related common one.
Disorder classification from fMRI connectivity usually relies on Transformers, whose cost grows quadratically with the number of brain regions and which ignore what is known about brain organisation. CP-SSM replaces self-attention with Mamba, a linear-time state-space model, and the feed-forward layer with a mixture of experts whose routing follows the core-periphery principle: core regions exchange information with every region, peripheral regions not with each other. It beat Transformer and graph baselines on autism (ABIDE, 1,009 subjects, AUROC 82.4 vs 79.6) and mild cognitive impairment (ADNI, 440 subjects, 80.7 vs 74.5), and its region weights highlight areas such as the precuneus in autism and the anterior cingulate in MCI.
3-hinge gyri, the points where three gyral crests meet, act as fine-scale hubs, but analysing them one node at a time ignores how they group into communities. This paper partitions each person's GyralNet into subnetworks with a graph network trained on a differentiable spectral-modularity objective that, unlike earlier relaxations, also uses node attributes: neighbourhood structural similarity with anatomical labels, and fibre-shape features from DTI. On 1,056 HCP subjects, combining both feature types gave lower conductance, higher modularity and more consistent subnetworks across people than either alone, at 5–10 seconds per brain, which makes community-level analysis of folding hubs practical.
Vision-language models usually cut brain scans into patches like ordinary photos, although brain disorders are network-level problems. This work treats each region's structural-connection profile from DTI as a token and aligns these sub-network tokens with clinical reports (demographics, APOE4, MMSE, CDR and doctors' notes) at two levels: cross-attention between sub-networks and words, and contrastive alignment between whole-brain and whole-report embeddings. On 418 ADNI subjects it separated MCI from controls with 92.9% accuracy, against 73.8% from imaging alone and 87.0% from text alone, and the alignment exposes readable links such as cingulate sub-networks with tau-related terms.
Multimodal LLMs could combine MRI with clinical notes for Alzheimer's diagnosis, but training them on 3D scans is expensive and medical datasets are small. Brain-Adapter places a small 3D convolutional bottleneck in front of a frozen medical multimodal LLM (M3D) to compress T1 MRI into its input space, aligns image and clinical-text embeddings with a CLIP-style loss, and fine-tunes only a few parameters. On 16,797 ADNI scans paired with clinical reports it classified Alzheimer's, MCI and normal controls with a macro F1 of 0.90, and with images alone it edged out 3D DenseNet121 and ResNet50: a cheap route for adapting foundation models to 3D neuroimaging.
Functional connectivity changes over the course of an fMRI scan, yet most studies of mild cognitive impairment treat it as static. This paper builds dynamic connectivity with sliding windows over 148 cortical regions and models it with a spatio-temporal Transformer (separate temporal and spatial blocks joined by global attention), plus a contrastive loss that pulls together subjects with the same diagnosis. On 345 ADNI subjects with 5-fold cross-validation it separated MCI from controls with 89.1% accuracy, against 82.4% without the contrastive term and 66.3% with static connectivity, showing that connectivity dynamics carry signal for catching the prodromal stage of Alzheimer's.
3-hinge gyri are stable folding hubs, but they are detected in each brain separately with no built-in correspondence across people, which blocks group studies built on them. This paper embeds every 3HG with a self-supervised Kolmogorov-Arnold Network autoencoder fed with multi-hop neighbourhood features weighted by structural similarity (degree sequences compared with dynamic time warping), and a reconstruction loss that up-weights the sparse non-zero entries. Trained on 1,064 HCP subjects (338,555 3HGs), the embeddings matched hubs across individuals with a single clear candidate in about 70% of cases, including frontal and parietal hubs where the earlier cortex2vec model failed.
Unsupervised Cortical Surface Registration Network for Aligning GyralNetJiale Cheng, Fenqiang Zhao, Dan Hu, Chao Cao, Zhengwang Wu, Xinrui Yuan, et al. (11 authors)
ISBI 2025
Cortical surface registration aligns curvature and convexity but ignores the gyral network (GyralNet) and its 3-hinge hubs, and no atlas of them existed. This unsupervised framework turns GyralNet and 3HGs into geodesic distance maps on the sphere, then jointly learns an age-conditioned atlas and a multi-level spherical U-Net that registers each brain to it, coarse to fine. On 1,089 HCP scans it aligned GyralNet better than FreeSurfer and Spherical Demons (map correlation 85.2% vs 72.1–76.0%, overlap IoU 23.0% vs 14.0–15.2%), produced a more detailed atlas, and does both in about 5 seconds instead of hours.
Ca-VQVAE: Cortical Folding Aware Numerical Representation of White-Matter StructureYanjun Lyu, Jing Zhang, Lu Zhang, Tong Chen, Xiaowei Yu, Minheng Chen, … Chao Cao, et al. (10 authors)
ISBI 2025
Tractography is noisy and fibre bundles vary between people, which makes white matter hard to compare across brains. CA-VQVAE compresses the shape of each local fibre bundle into a discrete code with a vector-quantised autoencoder, and adds contrastive labels from nearby cortical folding landmarks (3-hinge gyri on a template brain) so bundles under the same fold land close together. On 500 HCP subjects it reconstructed bundle descriptors more accurately than a VAE or vanilla VQ-VAE (MSE 1.14 vs 17.41 and 1.53), and its codes picked out corresponding cortical areas across subjects, giving white matter a compact numerical description tied to cortical folding.
20242 papers
Gyri vs. Sulci: Core-Periphery Organization in Functional Brain NetworksXiaowei Yu, Lu Zhang, Chao Cao, Tong Chen, Yanjun Lyu, Jing Zhang, Tianming Liu, Dajiang Zhu
MICCAI 2024
Gyri and sulci differ in anatomy and wiring, but how they divide the work of brain function was unclear. Using a Twin-Transformer that separates HCP task fMRI from seven tasks into spatial brain networks and their time courses, the paper measures how often gyral and sulcal voxels take part in the same network: gyri with gyri most, gyri with sulci next, sulci with sulci least, in every task and threshold tested (for example 0.44–0.54, 0.30–0.32 and 0.17–0.24). It frames gyri as a densely connected core that gathers and distributes information and sulci as a periphery for local processing, a basic organising principle for brain networks.
Mild Cognitive Impairment Classification Using A Novel Finer-Scale Brain ConnectomeYanjun Lyu, Lu Zhang, Xiaowei Yu, Chao Cao, Tianming Liu, Dajiang Zhu
ISBI 2024
Brain networks normally use atlas regions as nodes, assuming each region is uniform inside. This paper builds a finer-scale connectome whose nodes are 77 3-hinge gyri per hemisphere, matched across subjects with cortex2vec embeddings and the Hungarian algorithm, and compares it with AAL-90 and Destrieux connectomes for telling MCI from controls on 421 ADNI subjects with six standard classifiers. Structural connectivity between 3HGs reached 0.86–0.91 accuracy, against 0.73–0.78 for Destrieux and 0.53–0.62 for AAL-90, and functional connectivity improved too, suggesting that folding hubs capture neurodegeneration that region averages smooth over.