UTXO is an acronym for unspent transaction output. Every Bitcoin transaction in every block contains at least one output, which details how many Bitcoins are owned by the given public key(s). Outputs are then spent by inputs of later transactions and typically must be unlocked with a digital signature (most commonly ECDSA in Bitcoin). Until an output is used as input in another transaction, this output is called a UTXO.
All UTXOs at a given time can be collected into a set, called the UTXO set. The UTXO set represents the current state of ownership of all Bitcoin in existence. The inputs of transactions that are to be added to the blockchain are checked against this set. If an input to a transaction is an output in the UTXO set, the input can be validated. The UTXO set is deterministic for every Bitcoin block, that is, every node who has validated the history of the Bitcoin ledger up to a particular block should have an identical UTXO set. This is required in order for nodes to reach consensus on the validity of inputs in new transactions.
Confusingly the term UTXO is also often used as shorthand to refer to the set. Generally it is differentiated by the preceeding word. When referring to the set it is common to call it “the UTXO”.
When a node receives a new transaction, one of the checks on the transaction is if the input is spending a UTXO in the set. If this check is passed, the output that is being spent is taken out of the UTXO set, and temporarily stored in a buffer. If the transaction is accepted into a block, the corresponding UTXO is deleted from this temporary storage. On the other hand, if the transaction that spends the UTXO is not included in a block, the UTXO is returned to the UTXO set as the output is still unspent. As soon as an output is spent, it is no longer in the UTXO set. This process prevents an output from being double-spent.
Provably unspendable outputs
There is a subset of outputs which are provably unspendable. These outputs, whilst ‘unspent’, are not included in the UTXO set, as they are provably unspendable and will never be needed to validate a future transaction. Prior to the Genesis upgrade in February 2020, one form of these outputs start with the OP_RETURN opcode in the ScriptPubKey field of a transaction. Prior to this upgrade, The OP_RETURN opcode ended the script execution and returned a failure. After the Genesis upgrade, this provably unspendable output will be unlocking scripts beginning with the opcodes OP_FALSE OP_RETURN.