Q:

AWS SageMaker notebook list tables using boto3 and PySpark

Having some difficulty executing the following code in AWS SageMaker. It is supposed to just list all of the tables in DynamoDB.
import boto3
resource = boto3.resource('dynamodb', region_name='xxxx')
response  = resource.tables.all()
for r in response:
    print(r.name)

If the SageMaker notebook kernel is set to "conda_python3" the code executes fine and the tables are listed out in the notebook as expected (this happens pretty much instantly).
However, if I set the kernel to "Sparkmagic (PySpark)" the same code infinitely runs and doesn't output the table list at all.
Does anyone know why this would happen for the PySpark kernel but not for the conda3 kernel? Ideally I need to run this code as part of a bigger script that relies on PySpark, so would like to get it working with PySpark.

A:

Figured out what the issue was, you need to end an endpoint to tour VPC for DyanmoDB.
To do this navigate to:

AWS VPC
Endpoints
Create Endpoint
Select the dynamodb service (will be type Gateway)
Select the VPC your Notebook is using

