pyspark.sql.DataFrame.executionInfo

property DataFrame.executionInfo

Returns a ExecutionInfo object after the query was executed.

The executionInfo method allows to introspect information about the actual query execution after the successful execution. Accessing this member before the query execution will return None.

If the same DataFrame is executed multiple times, the execution info will be overwritten by the latest operation.

New in version 4.0.0.

Returns

An instance of ExecutionInfo or None when the value is not set yet.

Notes

This is an API dedicated to Spark Connect client only. With regular Spark Session, it throws an exception.