Fundamental Security Part Four – Permissions Checking

July 2023 – Oracle 19C

So in the previous post we talked about identifying unused and historic accounts. Old accounts should be removed, but what are the capabilities of the existing users who are accessing the systems?

A surprisingly large amount of applications request that the schema owner, and maybe the primary application connection, has DBA-level rights. This is always excessive and should be avoided.

Most DBA’s don’t need DBA rights most of the time. It makes a lot of sense to have a company-specific “DBA” role with minimal permissions. Most day-to-day checking tasks are read only. It is sensible to have a fairly low privilege account for day-to-day access and an admin account (preferable accessed via raising a ticket in response to an incident or change request) for when high privilege change is needed.

Your system permissions should be allocated on the principle of least privilege. Only grant those privileges and resources explicitly and legitimately required to perform the function.

Who Has DBA?

SELECT * FROM dba_role_privs 
WHERE granted_role = 'DBA'
ORDER BY grantee;

GRANTEE                        GRANTED_ROLE       ADM DEL DEF COM INH
------------------------------ ------------------ --- --- --- --- ---
APP_SCHEMA                     DBA                NO  NO  YES NO  NO
ERIK                           DBA                NO  NO  YES NO  NO
MARTIN                         DBA                NO  NO  YES NO  NO
NEIL                           DBA                NO  NO  YES NO  NO
TIM                            DBA                NO  NO  YES NO  NO
SYS                            DBA                YES NO  YES YES YES
SYSTEM                         DBA                NO  NO  YES YES YES

Identify who really needs DBA privs? NEIL the DBA does, but does ERIK the Developer need such high level privs? This is unlikely, even in a dedicated Development database.

However, you do need to be aware of other privileges. A CIS-level audit will pick up on DBA privs, but will not necessarily spot who has the IMP_FULL_DATABASE role granted to them (or, more powerfully, DATAPUMP_IMP_FULL_DATABASE). These roles contain very high level privileges such as ALTER DATABASE, DELETE ANY TABLE, and BECOME USER which should be highly restricted.

SELECT * FROM dba_role_privs 
WHERE granted_role = 'IMP_FULL_DATABASE'
ORDER BY grantee;

GRANTEE                        GRANTED_ROLE       ADM DEL DEF COM INH
------------------------------ ------------------ --- --- --- --- ---
DATAPUMP_IMP_FULL_DATABASE     IMP_FULL_DATABASE  NO  NO  YES YES YES
DBA                            IMP_FULL_DATABASE  NO  NO  YES YES YES
SCOTT                          IMP_FULL_DATABASE  NO  NO  YES NO  NO
SYS                            IMP_FULL_DATABASE  YES NO  YES YES YES

PUBLIC Grants

The above roles are fairly well known. A lesser known risk is the amount of procedures with EXECUTE permissions granted to PUBLIC. Anyone with access to the database can use these. However, there are over 2,500 such permissions in Oracle 19.13. The CIS Security standard helps to identify some of the higher risk grants, to packages such as UTL_TCP (which allows access to [corrupt] the TCP stream), UTL_HTTP (which could send information to external websites), DBMS_SQL (which, if used incorrectly, allows SQL Injection attacks), and DBMS_RANDOM (because it’s not very random and shouldn’t be used for any serious work!)

Sample of potentially insecure packages granted to public:

Network Security
DBMS_LDAP 
UTL_INADDR 
UTL_TCP 
UTL_MAIL 
UTL_SMTP 
UTL_DBWS 
UTL_ORAMTS 
UTL_HTTP 
HTTPURITYPE 

File Security
DBMS_ADVISOR
DBMS_LOB
UTL_FILE

Encryption
DBMS_CRYPTO
DBMS_OBFUSCATION_TOOLKIT
DBMS_RANDOM

Java
DBMS_JAVA
DBMS_JAVA_TEST

Scheduler
DBMS_SCHEDULER
DBMS_JOB

SQL Injection Helpers
DBMS_SQL
DBMS_XMLGEN
DBMS_XMLQUERY
DBMS_XLMSTORE
DBMS_XLMSAVE
DBMS_REDACT

(please review the CIS Standards for a list of the packages in your database release)

These grants can be seen in DBA_TAB_PRIVS (with the privilege of EXECUTE – the view isn’t only for tables)
It is a good practice to remove the grant of all of these packages from PUBLIC
[REVOKE EXECUTE ON <pkg> FROM PUBLIC; ]

WARNING: If you revoke these privileges from PUBLIC, you may find that your application stops working. You need to identify if any of the above packages are used and grant them explicitly to only the accounts which need them. The most common grant where you would need to do this would be for DBMS_LOB, but any of the above packages may be used by your apps or users. You need to discover which are being used (running checks against the application source code and view DBA_SOURCE, as well as monitoring the contents of GV$SQL and DBA_HIST_SQLTEXT are good starting points for this.)

Dangerous Grants

There are a number of very powerful and sensitive tables and procedures within Oracle where we need to monitor the grants carefully. Access to these is not granted by default, and we need to ensure this does not happen by accident!

Powerful Procedures
DBMS_BACKUP_RESTORE
DBMS_FILE_TRANSFER
DBMS_SYS_SQL
DBMS_REPCAT_SQL_UTL
INITJVMAUX
DBMS_AQADM_SYS
DBMS_STREAMS_RPC
DBMS_PRVTAQIM
LTADM
DBMS_IJOB
DBMS_PDB_EXEC_SQL

Sensitive Tables
CDB_LOCAL_ADMINAUTH$
DEFAULT_PWD$
ENC$
HISTGRM$
HIST_HEAD$
LINK$
PDB_SYNC$
SCHEDULER$_CREDENTIAL
USER$
USER_HISTORY$
XS$VERIFIERS

You may be wondering what is so sensitive about these tables and procedures. The DBMS_SYS_SQL package could allow a user to run code as a different user without entering valid credentials. Table HISTGRM$ contains samples of (or maybe all) data from table columns. Access to these objects is a notable security risk which needs to be taken seriously.

Here’s a useful bit of SQL which will look for access to these objects

SELECT owner, table_name, grantee, privilege, type FROM dba_tab_privs
WHERE table_name IN ('DBMS_LDAP', 'UTL_INADDR', 'UTL_TCP', 'UTL_MAIL', 'UTL_SMTP', 
'UTL_DBWS', 'UTL_ORAMTS', 'UTL_HTTP', 'HTTPURITYPE', 'DBMS_ADVISOR', 'DBMS_LOB', 
'UTL_FILE', 'DBMS_CRYPTO', 'DBMS_OBFUSCATION_TOOLKIT', 'DBMS_RANDOM', 'DBMS_JAVA',
'DBMS_JAVA_TEST', 'DBMS_SCHEDULER', 'DBMS_JOB', 'DBMS_SQL', 'DBMS_XMLGEN', 
'DBMS_XMLQUERY', 'DBMS_XLMSTORE', 'DBMS_XLMSAVE', 'DBMS_REDACT', 
'CDB_LOCAL_ADMINAUTH$', 'DEFAULT_PWD$', 'ENC$', 'HISTGRM$', 'HIST_HEAD$', 'LINK$', 
'PDB_SYNC$', 'SCHEDULER$_CREDENTIAL', 'USER$', 'USER_HISTORY$', 'XS$VERIFIERS', 'DBMS_BACKUP_RESTORE', 'DBMS_FILE_TRANSFER', 'DBMS_SYS_SQL', 'DBMS_REPCAT_SQL_UTL', 'INITJVMAUX', 'DBMS_AQADM_SYS', 'DBMS_STREAMS_RPC', 'DBMS_PRVTAQIM','LTADM', 'DBMS_IJOB', 'DBMS_PDB_EXEC_SQL')
ORDER BY owner,table_name;


OWN TABLE_NAME                GRANTEE              PRIVILEGE  TYPE
SYS DBMS_ADVISOR              PUBLIC               EXECUTE    PACKAGE
SYS DBMS_BACKUP_RESTORE       SYSBACKUP            EXECUTE    PACKAGE
SYS DBMS_FILE_TRANSFER        EXECUTE_CATALOG_ROLE EXECUTE    PACKAGE
SYS DBMS_IJOB                 IMP_FULL_DATABASE    EXECUTE    PACKAGE
SYS DBMS_JAVA                 PUBLIC               EXECUTE    PACKAGE
SYS DBMS_JOB                  PUBLIC               EXECUTE    PACKAGE
SYS DBMS_LDAP                 PUBLIC               EXECUTE    PACKAGE
SYS DBMS_LOB                  WMSYS                EXECUTE    PACKAGE
SYS DBMS_LOB                  PUBLIC               EXECUTE    PACKAGE
SYS DBMS_LOB                  ORDPLUGINS           EXECUTE    PACKAGE
SYS DBMS_LOB                  ORDSYS               EXECUTE    PACKAGE
SYS DBMS_OBFUSCATION_TOOLKIT  PUBLIC               EXECUTE    PACKAGE
SYS DBMS_PDB_EXEC_SQL         XDB                  EXECUTE    PROCEDURE
SYS DBMS_PDB_EXEC_SQL         AUDSYS               EXECUTE    PROCEDURE
SYS DBMS_RANDOM               ORDSYS               EXECUTE    PACKAGE
SYS DBMS_RANDOM               PUBLIC               EXECUTE    PACKAGE
SYS DBMS_REDACT               IMP_FULL_DATABASE    EXECUTE    PACKAGE
SYS DBMS_REDACT               EXECUTE_CATALOG_ROLE EXECUTE    PACKAGE
SYS DBMS_SCHEDULER            PUBLIC               EXECUTE    PACKAGE
SYS DBMS_SCHEDULER            MDSYS                EXECUTE    PACKAGE
SYS DBMS_SCHEDULER            AUDSYS               EXECUTE    PACKAGE
SYS DBMS_SQL                  AUDSYS               EXECUTE    PACKAGE
SYS DBMS_SQL                  PUBLIC               EXECUTE    PACKAGE
SYS DBMS_SQL                  DVSYS                EXECUTE    PACKAGE
SYS DBMS_SQL                  ORDSYS               EXECUTE    PACKAGE
SYS DBMS_STREAMS_RPC          EXECUTE_CATALOG_ROLE EXECUTE    PACKAGE
SYS DBMS_XMLGEN               PUBLIC               EXECUTE    PACKAGE
SYS DBMS_XMLQUERY             PUBLIC               EXECUTE    PACKAGE
SYS HISTGRM$                  NEIL                 SELECT     TABLE
SYS HTTPURITYPE               PUBLIC               EXECUTE    TYPE
SYS UTL_FILE                  WMSYS                EXECUTE    PACKAGE
SYS UTL_FILE                  PUBLIC               EXECUTE    PACKAGE
SYS UTL_FILE                  ORDPLUGINS           EXECUTE    PACKAGE
SYS UTL_FILE                  ORDSYS               EXECUTE    PACKAGE
SYS UTL_HTTP                  ORDPLUGINS           EXECUTE    PACKAGE
SYS UTL_HTTP                  PUBLIC               EXECUTE    PACKAGE
SYS UTL_INADDR                PUBLIC               EXECUTE    PACKAGE
SYS UTL_INADDR                DVSYS                EXECUTE    PACKAGE
SYS UTL_SMTP                  PUBLIC               EXECUTE    PACKAGE
SYS UTL_TCP                   PUBLIC               EXECUTE    PACKAGE

High Level Permissions

Another source of risk are various high level permissions allowing access to objects outside of your own user. The obvious ones are the “ANY” privileges, such as SELECT ANY TABLE, READ ANY TABLE and EXECUTE ANY PROCEDURE. and GRANT ANY PRIVILEGE.

You need to identify which users have these privileges and consider if a focussed role granting the necessary levels of permissions is a viable solution.

SELECT dsp.grantee
      ,dsp.privilege
      ,dsp.admin_option
      ,decode(dr.role,null,'USER','ROLE') user_or_role
      ,LISTAGG(DISTINCT drp.grantee,', ' ON OVERFLOW TRUNCATE) WITHIN GROUP (ORDER BY drp.grantee) GRANTED_TO
  FROM dba_sys_privs dsp LEFT OUTER JOIN dba_roles dr ON (dsp.grantee=dr.role)
                         LEFT OUTER JOIN dba_role_privs drp ON (dsp.grantee=drp.granted_role)
 WHERE dsp.privilege LIKE '%ANY%'
    -- exclude Oracle users
   AND dsp.grantee NOT IN (SELECT du.username FROM dba_users du WHERE oracle_maintained = 'Y')
 GROUP BY dsp.grantee,dsp.privilege,dsp.admin_option,decode(dr.role,null,'USER','ROLE')
 ORDER BY user_or_role,dsp.grantee,dsp.privilege
/

GRANTEE                    PRIVILEGE            ADM USER GRANTED_TO
-------------------------- -------------------- --- ---- ------------------------------
AQ_ADMINISTRATOR_ROLE      DEQUEUE ANY QUEUE    YES ROLE GSMADMIN_ROLE, GSMCATUSER, MDS
AQ_ADMINISTRATOR_ROLE      ENQUEUE ANY QUEUE    YES ROLE GSMADMIN_ROLE, GSMCATUSER, MDS
AQ_ADMINISTRATOR_ROLE      MANAGE ANY QUEUE     YES ROLE GSMADMIN_ROLE, GSMCATUSER, MDS
AUDIT_ADMIN                AUDIT ANY            NO  ROLE SYS
...
DATAPUMP_IMP_FULL_DATABASE GRANT ANY ROLE       NO  ROLE DBA, GSMADMIN_INTERNAL, SYS
DATAPUMP_IMP_FULL_DATABASE SELECT ANY TABLE     NO  ROLE DBA, GSMADMIN_INTERNAL, SYS
DBA                        ALTER ANY ASSEMBLY   NO  ROLE APP_SCHEMA, DODGY_DBA, NEIL,
...
SCHEDULER_ADMIN            EXECUTE ANY CLASS    YES ROLE DBA, SYS
SCHEDULER_ADMIN            EXECUTE ANY PROGRAM  YES ROLE DBA, SYS
APP_USER                   DELETE ANY TABLE     NO  USER 
APP_USER                   INSERT ANY TABLE     NO  USER 
APP_USER                   SELECT ANY TABLE     NO  USER 
APP_USER                   UPDATE ANY TABLE     NO  USER 
NEIL2                      SELECT ANY TABLE     NO  USER 

Please note that this is just the ‘%ANY%’ privileges. You need to be aware of users with other very high level privileges such as ALTER DATABASE, ALTER SYSTEM, AUDIT SYSTEM, ALTER USER, DROP USER and DROP PUBLIC SYNONYM. Hint: a good starting point is to remove the ‘dsp.privilege LIKE ‘%ANY%’‘ predicate and look to see what permissions SYS has granted to it. Privileges are added and removed with every Oracle release and you need to understand what they are doing.

Understand who has access to what, and if that access is justified. If not, determine how to reduce the access safely, without compromising your system availability whilst tightening your security.

Next we will talk about Observability. Things change. You need to be watching, regularly, to catch the change and ensure it’s valid.

Fundamental Security Part Three – Unused and Historic Accounts

June 2023 – Oracle 19C

So in the previous post we talked about implementing password complexity. Complex passwords are fine, but what when you experience change. For example, someone leaves the company to become a professional skydiver. Do you remove their database accounts? Are you even informed by HR that someone with database access has left?

If you have implemented user profiles appropriately, the account – left unused – will lock eventually, which is a good start. Pretty much every organization I investigate has old accounts hanging around in every database. You need to identify these accounts and remove them.

Fortunately, within recent versions of Oracle (12C+), account access is recorded in LAST_LOGIN column in the DBA_USERS view.

Users with SYSDBA/SYSBACKUP/etc privs do not update DBA_USERS (well, the underlying table SYS.USER$) as they must be able to connect to a database which is not available. They should update V$PWFILE_USERS.LAST_LOGIN instead of DBA_USERS, but as of my last test on 19.15, they do not.

If we run a query against DBA_USERS, it can be instructive to see what it going on. Reviewing the ACCOUNT_STATUS, CREATED data and LAST_LOGIN, we should be able to make some preliminary decisions about which accounts should continue to exist, and which accounts should be deleted.

DBA_USERS

SELECT username
      ,oracle_maintained
      ,account_status status
      ,created
      ,nvl(last_login,'never') last_login
  FROM dba_users ORDER BY 2, 1;

USERNAME   O STATUS  CREATED    LAST_LOGIN  my comments
APP_SCHEMA N OPEN    2019-04-18 2022-01-01  <- a schema owner
APP_USER   N OPEN    2019-04-18 2023-06-15  <- the main application user
BKP_USR    N OPEN    2019-04-18             <- no last login. WHY?
MARTIN     N OPEN    2020-11-16 2021-04-20  <- He retired
NEIL       N OPEN    2021-11-15 2023-06-15  <- me!
NIALL      N OPEN    2019-10-02 2020-12-12  <- off skydiving?
SCOTT      N LOCKED  2019-11-15 never       <- should this test schema exist?
TIM        N OPEN    2019-11-17 never       <- old and unused! Delete!
.
AUDSYS     Y LOCKED  2019-04-17 never
CTXSYS     Y LOCKED  2019-04-17 never
.
SYSRAC     Y LOCKED  2019-04-17 never
SYSTEM     Y OPEN    2019-04-17 2021-11-16  <- Do we use this? Should it be locked?
WMSYS      Y LOCKED  2019-04-17 never
XDB        Y LOCKED  2019-04-17 never
XS$NULL    Y LOCKED  2019-04-17 never

Take note of the ORACLE_MAINTAINED flag. You should not delete any accounts with this set to "Y", except under the supervision of Oracle Support, as this may cause system instability or even break the database.

Lets look at V$PWFILE_USERS too

$ sqlplus bkp_usr/oracle as sysbackup
Connected.
SQL > show user
USER is "SYSBACKUP"


SELECT username
      ,sysdba
      ,sysoper
      ,sysasm
      ,sysbackup
      ,sysdg
      ,syskm
      ,account_status status
      ,last_login
--    ,lock_date
--    ,expiry_date
--    ,authentication_type
--    ,common
      ,con_id
FROM  V$PWFILE_USERS
order by 1
/

USERNAME   SYSDB SYSOP SYSAS SYSBA SYSDG SYSKM STATUS  LAST_LOGIN         CON_ID
BKP_USR    FALSE FALSE FALSE TRUE  FALSE FALSE OPEN                            3           
SYS        TRUE  TRUE  FALSE FALSE FALSE FALSE OPEN                            0                                           

We can see from the above that BKP_USR is not registering logins as a sys privileged accounts (sysbackup), although it is used daily… We can see that it is indeed SYSBACKUP privs, but there’s no entry in LAST_LOGIN in either view.

We need to periodically manually review accounts in the database for anomalies and confirm that they are all correct. Create a regular report for this – there’s a good chance Security may like a copy.


Of course, there is an alternative to managing user populations in every database individually. From 18C, Oracle introduced Centrally Managed UsersCMU. This allows a database to directly authenticate accounts against Microsoft AD, without using an intermediary service such as OID or Oracle Enterprise User Security. I would recommend using MOS Document: 2462012.1 as a starting point, and initially integrating administrative users before opening up to the wider user population. Maybe not for schemas, though?

Next, we will check who has what permissions, and if there are any security surprises in there: Fundamental Security Part Four – Permissions Checking

Fundamental Security Part Two – Password Complexity and Defaults

June 2023 – Oracle 19C

So in the previous post we saw the creation of a PROFILE to add controls on user passwords, but we did not specify password complexity. By default, this is set to NULL in the profile, meaning no password complexity is enforced. This is a bad practice and must be changed.

PROFILE          RESOURCE_NAME             LIMIT
---------------- ------------------------- ---------
DEFAULT          PASSWORD_VERIFY_FUNCTION  NULL

Oracle provides a number of built-in password verification functions to set minimum length, complexity and the amount of character change required when entering a new password. Their details are as follows (Note that in 19C, VERIFY_FUNCTION_11G  and ORA12C_VERIFY_FUNCTION  are deprecated but still available)

Function NameLengthUpper Lower Numeric Special Differs-By
ORA12C_STIG_VERIFY_FUNCTION1511118
ORA12C_STRONG_VERIFY_FUNCTION922224
ORA12C_VERIFY_FUNCTION 811103
VERIFY_FUNCTION_11G 101103

Of the 4 build-in options, the ORA12C_STIG_VERIFY_FUNCTION is the “best” due to the level of complexity but mainly the length. Passwords need to be long to make them difficult to decrypt or crack. Having a guaranteed mix of upper case, lower case, numeric and special characters mean that there are a lot of possible options for each character position (over 70).

It is possible to write your own password complexity function, and Oracle provide the code for the above options in clear text in the file : $ORACLE_HOME/rdbms/admin/catpvf.sql


Simple example of creating your own custom password verification function based upon the built-in Oracle functions ora_complexity_check and ora_string_distance:


CREATE OR REPLACE FUNCTION custom_password_verify (
    username     VARCHAR2
  , password     VARCHAR2
  , old_password VARCHAR2
) RETURN BOOLEAN IS
    differ  INTEGER;
BEGIN
    IF NOT ora_complexity_check(
                               password
                             , chars => 15
                             , uppercase => 1
                             , lowercase => 1
                             , digit => 1
                             , special => 1
           ) THEN
        RETURN ( false );
    END IF;
    -- Check if the password differs from the previous password by n characters
    IF old_password IS NOT NULL THEN
        differ := ora_string_distance(old_password, password);
        IF differ < 8 THEN
            raise_application_error(-20000, 'password is too similar to previous password');
        END IF;

    END IF;
RETURN ( true );
END;
/

ALTER PROFILE default LIMIT PASSWORD_VERIFY_FUNCTION custom_password_verify;

NOTE: The function ORA_STRING_DISTANCE computes the Levenshtein distance between 2 words, which is the minimum number of single character edits required to change one word into the other.

It can be a good idea to match the complexity required within the Oracle database passwords with the complexity required with your corporate AD or LDAP password validation.

More on using AD or LDAP to authenticate within Oracle in another blog post.

WARNING

The PASSWORD_VERIFY_FUNCTION is a very powerful function which needs to be protected. Given that passwords are passed to it in clear text, and it is possible to subvert the procedure to perform illicit actions (such as keeping a backdoor permanently unlocked or recording everyones password) with the function, care should be taken to review and monitor the function for change and unexpected code.

Default Passwords

There are many standard Oracle accounts in existence, some of which are intrinsic to the running of the system such as SYS and SYSRAC. Some are automatically installed in the database, such as OUTLN or ORDDATA, and some are are simply well know, such as SCOTT or HR. These accounts have default passwords, which are commonly known to every hacker. Oracle provides as view which will display any accounts with a default password, called: DBA_USERS_WITH_DEFPWD

Installing a vanilla Oracle19.3 database and running a select yields the following results:


SQL > SELECT * FROM dba_users_with_defpwd;

USERNAME                       PRODUCT
------------------------------ ------------------------------
SYS
SYSTEM
CTXSYS

SQL > conn CTXSYS/CTXSYS
ERROR:
ORA-28000: The account is locked.

As we can see, the system believed SYS and SYSTEM have the default passwords, and CTXSYS definitely does have the default password! Fortunately, it is also locked by default, but it might be a good idea to change the password immediately.

However, SYS and SYSTEM force you to change the password on installation. There’s a minor bug which means the SHA1 hash which is used to identify default passwords is not updated at installation time. It is a good idea to re-change the SYS and SYSTEM password after install to update this and demonstrate to any audit or compliance officer running the above SELECT that the password is, indeed, not set to the default.

For more details about the underlying mechanisms which are used to allow this view to work, please check out Pete Finnegans blog post here

Now, lets look at old users…

Adding a DEFAULT column in 12C

I was at a talk recently, and there was an update by Jason Arneil about adding columns to tables with DEFAULT values in Oracle 12C. The NOT NULL restriction has been lifted and now Oracle cleverly intercepts the null value and replaces it with the DEFAULT meta-data without storing it in the table. To repeat the 11G experiment I ran recently:

 

SQL> alter table ncha.tab1 add (filler_default char(1000) default 'EXPAND' not null);
Table altered.

SQL> select table_name,num_rows,blocks,avg_space,avg_row_len 
      from user_tables where table_name = 'TAB1';
TABLE_NAME NUM_ROWS       BLOCKS  AVG_SPACE AVG_ROW_LEN
---------- ---------- ---------- ---------- -----------
TAB1            10000       1504          0        2017


In both releases we then issue:
SQL> alter table ncha.tab1 modify (filler_default null);
Table altered.


IN 11G
SQL> select table_name,num_rows,blocks,avg_space,avg_row_len
      from user_tables where table_name = 'TAB1';

TABLE_NAME NUM_ROWS       BLOCKS  AVG_SPACE AVG_ROW_LEN
---------- ---------- ---------- ---------- -----------
TAB1            10000       3394          0        2017

BUT IN 12C
SQL> select table_name,num_rows,blocks,avg_space,avg_row_len
      from user_tables where table_name = 'TAB1';
TABLE_NAME NUM_ROWS       BLOCKS  AVG_SPACE AVG_ROW_LEN
---------- ---------- ---------- ---------- -----------
TAB1            10000       1504          0        2017

So, as we can see, making the column NULLABLE in 12C didn’t cause it to go through and update every row in the way it must in 11G. It’s still a chained-row update accident waiting to happen, but its a more flexible accident 🙂

However, I think it’s worth pointing out that you only get “free data storage” when you add the column. When inserting a record, simply having a column with a DEFAULT value means that the DEFAULT gets physically stored with the record if it is not specified. The meta-data effect is ONLY for subsequently added columns with DEFAULT values.

SQL> create table ncha.tab1 (pk number, c2 timestamp, filler char(1000), filler2 char(1000) DEFAULT 'FILLER2' NOT NULL) pctfree 1;
Table created.

SQL> alter table ncha.tab1 add constraint tab1_pk primary key (pk);
Table altered.

Insert 10,000 rows into the table, but not into FILLER2 with the DEFAULT
SQL> insert into ncha.tab1 (pk, c2, filler) select rownum id, sysdate, 'A' from dual connect by level <= 10000;
commit;
Commit complete.

Gather some stats and have a look after loading the table. Check for chained rows at the same time.
SQL> exec dbms_stats.gather_table_stats('NCHA','TAB1',null,100);
PL/SQL procedure successfully completed.

SQL> select table_name,num_rows,blocks,avg_space,avg_row_len
     from user_tables where table_name = 'TAB1';

TABLE_NAME   NUM_ROWS	  BLOCKS  AVG_SPACE AVG_ROW_LEN
---------- ---------- ---------- ---------- -----------
TAB1		10000	    3394	  0	   2017

For a bit of fun, I thought I would see just how weird the stats might look if I played around with adding defaults

SQL> drop table ncha.tab1;
Table dropped.

SQL> create table ncha.tab1 (pk number) pctfree 1;
Table created.

SQL> alter table ncha.tab1 add constraint tab1_pk primary key (pk);
Table altered.

Insert 10,000 rows into the table

SQL> insert into ncha.tab1 (pk) select rownum id from dual connect by level <= 10000;
commit;
Commit complete.

Gather some stats and have a look after loading the table. Check for chained rows at the same time.
SQL> exec dbms_stats.gather_table_stats('NCHA','TAB1',null,100);

PL/SQL procedure successfully completed.

SQL> select table_name,num_rows,blocks,avg_space,avg_row_len
  2    from user_tables
  3   where table_name = 'TAB1';

TABLE_NAME   NUM_ROWS	  BLOCKS  AVG_SPACE AVG_ROW_LEN
---------- ---------- ---------- ---------- -----------
TAB1		10000	      20	  0	      4

Now lets add a lot of defaults
SQL> alter table ncha.tab1 add (filler_1 char(2000) default 'F1' not null, filler_2 char(2000) default 'F2' null, filler_3 char(2000) default 'F3', filler_4 char(2000) default 'how big?' null );
Table altered.

Gather some stats and have a look after adding the column. Check for chained rows at the same time.
SQL> exec dbms_stats.gather_table_stats('NCHA','TAB1',null,100);

PL/SQL procedure successfully completed.

SQL> select table_name,num_rows,blocks,avg_space,avg_row_len
  2    from user_tables
  3   where table_name = 'TAB1';

TABLE_NAME   NUM_ROWS	  BLOCKS  AVG_SPACE AVG_ROW_LEN
---------- ---------- ---------- ---------- -----------
TAB1		10000	      20	  0	   8008

10,000 rows with an AVG_ROW_LEN of 8008, all in 20 blocks. Magic!

Just to finish off, lets update each DEFAULT column so the table expands….

SQL> select filler_1, filler_2, filler_3, filler_4,count(*) from ncha.tab1 group by filler_1,filler_2,filler_3,filler_4;

FILLER_1   FILLER_2   FILLER_3	 FILLER_4     COUNT(*)
---------- ---------- ---------- ---------- ----------
F1	   F2	      F3	 how big?	 10000

So it's all there. The metadata is intercepting the nulls and converting them to the default on the fly, rather than storing them in the blocks.
So what happens if we actually UPDATE the table?

SQL> update ncha.tab1 set filler_1 = 'EXPAND', filler_2 = 'EXPAND', filler_3='EXPAND', filler_4='THIS BIG!';
10000 rows updated.

SQL> select filler_1, filler_2, filler_3, filler_4,count(*) from ncha.tab1 group by filler_1,filler_2,filler_3,filler_4;

FILLER_1   FILLER_2   FILLER_3	 FILLER_4     COUNT(*)
---------- ---------- ---------- ---------- ----------
EXPAND	   EXPAND     EXPAND	 THIS BIG!	 10000

Gather some stats and have a look after the update, checking for chained rows at the same time.
SQL> exec dbms_stats.gather_table_stats('NCHA','TAB1',null,100);

PL/SQL procedure successfully completed.

SQL> select table_name,num_rows,blocks,avg_space,avg_row_len
     from user_tables where table_name = 'TAB1';

TABLE_NAME   NUM_ROWS	  BLOCKS  AVG_SPACE AVG_ROW_LEN
---------- ---------- ---------- ---------- -----------
TAB1		10000	   19277	  0	   8010

SQL> 
SQL> analyze table tab1 list chained rows into chained_rows;

Table analyzed.

SQL> select count(*) CHAINED_ROWS from chained_rows;

CHAINED_ROWS
------------
       10000

Yep. That’s bigger.