Troubleshooting common core services
Use these workarounds to troubleshoot problems that you might encounter with Flight service or other issues in common core services.
Can't load large data sets if the notebook environment doesn't have enough memory
You receive an error that states the notebook kernel is restarting because the kernel died while running the generated code that loads data from a connection to your notebook. You receive a "FlightTimedOutError" when you run the generated code. This timeout error occurs only if the generated code uses Flight service.
Cause
The reason is that you are trying to load a large amount of data to your notebook in a runtime environment with limited memory.
When you load large data sets into a specific data structure in a notebook, you typically need more memory than the size of the source data on disk. The amount of memory depends directly on the target data structure and the type of data. Additional memory is required for intermediate data structures that are created during the data transfer process from the source, depending on the transfer mechanism, such as Flight service, JDBC, or others.
Resolving the problem
Make sure that you are running your notebook in an environment runtime that is large enough to load the data sets you want to work with. If the provided environments aren't large enough, you can create a custom environment template.