Why can PL/pgSQL functions have side effect, while SQL functions can't?

Question

PostgreSQL document says:

The entire body of a SQL function is parsed before any of it is executed. While a SQL function can contain

Answer 1

Plpgsql functions are parsed and syntax-checked at definition time, then at first execution a plan is generated.

https://www.postgresql.org/docs/current/static/plpgsql-implementation.html#PLPGSQL-PLAN-CACHING

then that plan is executed with the given parameters.

Temporary files seem to work as expected, except those that already exist on the first execution.

As mentioned in there use of dynamic SQL (EXECUTE) is a way to foil the planner allowing access to arbitrary tables.

https://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-EXECUTING-DYN

Answer 2

You bolded the key sentence in the manual yourself:

The entire body of a SQL function is parsed before any of it is executed.

Also read about The Parser Stage in the manual.

It consists of two major parts: the parser and the transformation process. Quoting the manual:

the transformation process takes the tree handed back by the parser as input and does the semantic interpretation needed to understand which tables, functions, and operators are referenced by the query.

If an SQL function contains these commands:

CREATE TABLE foo (...);
INSERT INTO foo VALUES(...);

Both statements are planned at virtually the same time (based on the same snapshot of the system catalogs). Hence, the INSERT cannot see the table "foo" presumably created with the previous CREATE command. That creates one of the following problems:

1. If there is no other table named "foo" in your search_patch (yet), Postgres complains when trying to create the function:

   ERROR:  relation "foo" does not exist
   

2. If another table named "foo" already exists in your search_patch (and you don't use conflicting column names), Postgres will plan the INSERT based on that pre-existing table. Typically that results in an error at execution time, if any values cause conflicts in the (wrong!) table. Or, with some bad luck, it might even write to that table without error message! Very sneaky bug.

That cannot happen with a PL/pgSQL function, because it treats SQL commands like prepared statements, planned and executed sequentially. So each statement can see objects created in previous statements.

Consequently, statements that are never visited are never even planned - unlike with SQL functions. And the execution plan for statements can be cached within the same session - also unlike SQL functions. Read details about plan caching in PL/pgSQL functions in the manual here.
Each approach has advantages for some use cases. Further reading:

• Difference between language sql and language plpgsql in PostgreSQL functions