Why is IS NOT NULL returning NULL values for a Varchar(max) in SQL Server?

Question

1. It looks like some NULL values are appearing in the list.
2. Some NULL valu

Answer 1

This is just an addition to McSim's answer using SQL Server Internals Viewer to look at the individual stages.

CREATE TABLE [dbo].[Test](
    [Id] [int] NOT NULL PRIMARY KEY ,
    [Value] [text] NULL)


INSERT INTO Test VALUES (1, '')

Row after initial insert

Text value after initial insert

update [Test] SET [Value] = null 

Row after update to NULL

This is identical to the row shown earlier so I haven't repeated the screenshot. Specifically the NULL_BITMAP does not get updated to reflect the new NULL value.

Text value after update to NULL

The Type bits have changed and Internals Viewer shows this as no longer containing a value for the Data column.

At this point running the following correctly returns no rows

SET STATISTICS IO ON
select [Id]
from [Test]
where [Value] is not null

So SQL Server must follow the text pointer and look at the value there to determine NULL-ability.

ALTER TABLE [Test] ALTER COLUMN [Value] varchar(max)

This is a metadata only change. Both the inrow and out of row data remain unchanged.

However at this point running the following incorrectly returns the row.

SET STATISTICS IO ON
select [Id]
from [Test]
where [Value] is not null

The output of STATISTICS IO

Scan count 1, logical reads 2, ... lob logical reads 1

shows that it still does actually follow the text pointer but presumably in the varchar(max) case there must be a different code path that incorrectly ends up taking the value from the NULL_BITMAP regardless (the value of which has never been updated since the initial insert).