Testing World

Pink Ninja Tester

Category: Python

Django: distinct

pinkninjatester

I know there is distinct() function in Django ORM. However, it is used differently for each type of database. In this case, my application is using PostgreSQL but our unit test is using SQLite. So yeah good luck with it.

Some example using distinct:

YourModel.objects.distinct() # distinct for all combinations of columns in the table
YourModel.objects.distinct("a_column") # distinct only for a specific column

But as I mentioned, that our unit test is using SQLite. And only the first example is working for other than PostgreSQL. If we use the second example for our sql lite, it will throw this error 

DISTINCT ON fields is not supported by this database backend

So, what should we do if we just want to do distinct for a specific column in other database other than PostgreSQL? Then we can use values() or value_list() such as:

YourModel.objects.values("a_column").distinct()

We probably think that it will be translated to 

SELECT DISTINCT "YourModel"."a_column" FROM "YourModel"

But surprisingly, it is not the case. So if YourModel has defined the ordering in its Meta. Then this ordering will be included in the select distinct query. Okay for example, YourModel has ordering to ‘modified’, then the modified will be included into the query such as below:

class YourModel(models.Model):
    a_column = models.TextField()
    modified = models.TextField()

    class Meta:
        ordering = [ 'modified', ]

YourModel.objects.values("a_column").distinct()

# will be translated to 
SELECT DISTINCT "YourModel"."a_column", "YourModel"."modified" FROM "YourModel" ORDER BY "YourModel"."modified"

So the solution/workaround is actually simple. We just need to add order_by with nothing on it. 

YourModel.objects.values("a_column").distinct().order_by()

# will be translated to 
SELECT DISTINCT "YourModel"."a_column" FROM "YourModel"

Happy coding!

Python: \ufeff solution

pinkninjatester

Have you ever had this problem when you open a csv file using python and it did not recognize the column name even though you are 3000% sure the column name is there. 

Example csv data which is save as my_example.csv

id;name;age
1;aaa;11
2;bbb;12
3;ccc;13

So then you tried to open it using python such as:

with open('my_example.csv') as csv_file:
    reader = csv.DictReader(csv_file, delimiter=";")
        for row in list(reader)[:5]:
            print(row['id')

But then you are surprised because it throws error “KeyError: id is not recognized on row[‘id’]”. So I changed the code from print(row[‘id’] to print(row). I did that just because I am curious how the ptyhon unpack the csv file. Surprisingly it shows like this below:

{'\ufeffid': '1', 'name': 'aaa', 'age': '11'}
{'\ufeffid': '2', 'name': 'bbb', 'age': '12'}
{'\ufeffid': '3', 'name': 'ccc', 'age': '13'}

What is this \ufeff ?????

So this is called BOM – Byte Order Mark and is used to tell the difference between big- and little-endian UTF-16 encoding.  And when this issue is most likely occur? Imagine you create a csv file via excel and then when you save it, you save it in UTF-8. Excel by default will put a signature on the file and afaik there is no way to avoid it. 

Oke, so it is encoding problem. And then we can solve it with adding encoding when we open the csv file. 

with open('my_example.csv', encoding='utf-8-sig') as csv_file:
    reader = csv.DictReader(csv_file, delimiter=";")
        for row in list(reader)[:5]:
            print(row['id')

And voila, it works again! But will it work if the file we are trying to open does not have this issue? It is! The utf-8-sig encoding will decode both utf-8-sig-encoded text and text encoded with the standard utf-8 encoding. 

Django: AttributeError – select_related

pinkninjatester 
AttributeError: 'YourObject' object has no attribute 'select_related'

If you received this error, first check how you build up your django query. I had this error because I mis-structured it. 

what I did:

MyModel.objects.filter(somequery=somevalue).select_related(myrelation)

But what I should do:

MyModel.objects.select_related(myrelation).filter(somequery=somevalue)

Python: context manager

pinkninjatester

Most python programmer unconsciously use context manager without knowing its name. Have you ever opened a file using python? It is a high chance that you use it already. Yes it is the with syntax.

Context manager takes care or resource management such as setup (open) a connection and clean it up when it is no longer used. As mentioned above, one example of it is when we open a file. We use this syntax below.

with open('file_path/file_name.extension', mode) as f:
    for line in f:
        print(line)
        # do something

Python – What is Python?

pinkninjatester

Yes! Let’s start learning python! To be honest, I have worked with python for some time, on and off.

At one time, a client asked me to maintain their test automation which was built in python, and do bug fixing on that. Easy peasy. At another time, a client asked me to build their test automation from scratch in python. Done!

So basically, it is a really easy language to work with. But also don’t forget that I have a background in other programming languages such as javascript, Java, .NET, VB, and many others. 

But then…the one million dollar question. Do we know what Python is? What is Python? No no, it is not the snake. Yeah, that snake is indeed pretty cute. But No! It is not the snake. 

Okay, Python is a programming language that is easy to learn. It is applicable to many types of applications such as testing, web application, machine learning, data science, desktop application,  and most important automation.

Well, that sounds more like its benefits instead of describing what python is. So, what is Python?

Right, Python is an interpreted language. The python interpreter will turn the source code, that you have built, into machine code line by line once at a time during the program’s execution. 

Hmm, is it not the same concept as another language such as Java or .NET? The difference may be that they are using a compiler instead of an interpreter. 

Well, with Python, the source code will be interpreted to source code line by line instead of as a whole at one time. Boom! Hmm okay, I might not know what it is exactly. Don’t blame me. I bet only a few people know what Python is. And bet that they care more about its advantages or its benefits.

Okay, so what are its benefits? Why Python?

Many! I have mentioned some of them above. First, it is easy to learn. Second, it is applicable to many fields from web applications to data science applications. On top of that, Python allows us to solve complex problems with less time and code. Moreover, Python has a large ecosystems, library, frameworks, and huge community. And don’t forget, Python developer is in high demand on for sure high salary. 

So what can you ask more? Let’s dig it!

Happy learning Python!