The Dunning-Kruger Effect – I don’t know

Interviewing people is always a rich source of anecdotes, or should that be anec-dont’s? Whatever.

When interviewing for a technical specialist, such as a senior Oracle DBA, it seems important to ask a number of technical questions to ascertain the competency the individual has in relation to Oracle and the technologies related to Oracle. This is where the Dunning-Kruger effect comes into play beautifully.

Kruger and Dunning proposed that, for a given skill, incompetent people will:

  1. tend to overestimate their own level of skill;
  2. fail to recognize genuine skill in others;
  3. fail to recognize the extremity of their inadequacy;
  4. recognize and acknowledge their own previous lack of skill, if they are exposed to training for that skill;

So, lets ask a technical question of the candidate:

Under what circumstances will the optimizer decide not to use an index?

Here’s 4 extremely confident actual answers from 4 different candidates. This was the extent of their answers, although some (but not all) did elaborate further:

  • when you do a count(*)
  • when the columns you are selecting are not indexed
  • I query using a table and build a SQL Profile to bypass the index
  • it depends on the join condition

All candidates were OCP certified with at least 10 years worth of Oracle experience on their CV’s, applying for a senior position. To quote Darwin: “Ignorance more commonly begets confidence than does knowledge”. I like to think that it’s our brains protecting us from ourselves.

I mean… how hard is it to say “I don’t know” ? For some nationalities, that would appear to be impossible. A long time ago I spent several weeks teaching programmers in a country whose native/official language was not English. Given I am from the North of England, there are some who would say that this is also the case for me, given my notable regional accent. The combination of my idiosyncratic dialect, combined with the unnamed-county’s locals’ inability to lose face by saying “What you talkin’ about Neil?”, like Gary Coleman might have, meant that the second week of training was spent going through exactly the same materials as the first week, as it took me that long to realise they didn’t have a clue what I was saying half of the time. If only one of them had been brave enough to say (or I had been a little smarter and realised) then it would have save a whole lot of pain. However, the locals might never come understand what happens when you get “your knickers in a twist.”

NOTE: I do appreciate that I could be a Dunning-Kruger “victim” and these blogs merely indicative of immense incompetence. I’ll let you decide. 😐

Industry Experience

I don’t get it. Why do so many jobs and contracts seem to insist upon having experience in a particular industry when, in the overwhelming majority of cases, the specific industry in which we work has no bearing upon the nature of our work.

I have worked across many industries, but each time I talk to a recruitment agent I get similar questions: “Have you worked in X industry?”, “I won’t put you forward for Y unless you have worked for Z”.

It’s the wrong question. Have I worked in Media? Investment Banking? Accountancy? Property? Logistics? It doesn’t matter. No, really. It doesn’t. I have worked in all of those industries and a few more besides, and the nature of the industry was largely irrelevant. A friend recently suggested that you need Investment Banking experience so you understand the inordinate bureaucracy and dreadful boredom that come with working for an Investment Bank. A little unkind, but I know where he’s coming from.

What is relevant is the type and nature of systems with which you are working. Are they mission critical? Zero downtime? Very High Transaction rate? Enormous Data Warehouses? Hundreds or Thousands of databases? These questions have relevance. A high transaction rate OLTP in a Bank is very similar to a high transaction rate OLTP Web Retailer.  The challenge with these systems is a different to that of an enormous data warehouse, but it’s still fundamentally an RDBMS. Data is data is data. We don’t need industry experience – it doesn’t help us in the same way as it helps Business Analysts or Project Managers or even Developers.

The recruitment problem for DBA’s is that recruiters don’t know the difference between OLTP and Data Warehousing; a large proportion simply keyword match (the great ones don’t! – and there are genuinely great recruiters out there, in small numbers) so you need to ensure you have all of the relevant keywords on your CV – I have even been asked to amend my CV to put Word and Excel on there! WTF? Unfortunately you also need to be careful, otherwise you’re probably getting job adverts sent through for Cobol Programmers, Websphere Guru’s and all manner of support and helpdesk staff. I removed IBM Assembler Programmer from my CV about 10 years ago, although I suspect there are not too many jobs left for that skill set now.