PhotoSwipe does not force any HTML markup, you have full control. Simplest markup is a list of thumbnails that link to large image, simplest example:

<a href="large-image.jpg">
    <img src="small-image.jpg" alt="Image description" />
</a>
...

If you have long caption that doesn’t fit in alt or it just contains HTML tags, you may use <figure> and <figcaption>:

<figure>
	<a href="large-image.jpg">
	    <img src="small-image.jpg" alt="Image description" />
	</a>
	<figcaption>Long image description</figcaption>
</figure>
...

You can go further and use Schema.org markup for ImageGallery and ImageObject, it should look like this:

<div itemscope itemtype="http://schema.org/ImageGallery">

	<figure itemprop="associatedMedia" itemscope itemtype="http://schema.org/ImageObject">
		<a href="large-image.jpg" itemprop="contentUrl">
		    <img src="small-image.jpg" itemprop="thumbnail" alt="Image description" />
		</a>

		<!-- optionally use this method to store image dimensions for PhotoSwipe -->
		<meta itemprop="width" content="300">
		<meta itemprop="height" content="600">

		<figcaption itemprop="caption description">
			Long image description

			<!-- optionally define copyright -->
			<span itemprop="copyrightHolder">Photo: AP</span>
		</figcaption>
	</figure>

	<figure itemprop="associatedMedia" itemscope itemtype="http://schema.org/ImageObject">
		<a href="large-image.jpg" itemprop="contentUrl">
		    <img src="small-image.jpg" itemprop="thumbnail" alt="Image description" />
		</a>
		<figcaption itemprop="caption description">Long image description</figcaption>
	</figure>

	...

</div>

If you don’t want thumbnails to be visible on page, e.g. you have 50 images in gallery and you show just first 3 thumbnails + link “view all images (50)”, you definitely should use Schema.org markup and you should have all 50 links (with text in contents of link element instead of thumbnail) in DOM (you may hide them with display:none). Example:

<div itemscope itemtype="http://schema.org/ImageGallery">

	<figure itemprop="associatedMedia" itemscope itemtype="http://schema.org/ImageObject">
		<a href="large-image-1.jpg" itemprop="contentUrl">
		    <figcaption itemprop="caption description">Long image description 1</figcaption>
		</a>
	</figure>

	<figure itemprop="associatedMedia" itemscope itemtype="http://schema.org/ImageObject">
		<a href="large-image-2.jpg" itemprop="contentUrl">
		    <figcaption itemprop="caption description">Long image description 2</figcaption>
		</a>
	</figure>

	...

</div>

In all above cases, large-image.jpg will be perfectly indexed. The caption element will be crawled even if you hide it with display:none, just keep the text relevant, non-spammy – don’t stuff it with keywords.

Additional recommendations

Know how this page can be improved? Please suggest an edit!